Mercurial > kallithea

annotate scripts/update-copyrights.py @ 6777:5dde581f703f stable

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
scripts: add update-copyrights.py This script will automate how the current attribution and copyright listings are generated, thus also enabling it to be maintained from repository history in the future. This doesn't change anything about who has or hasn't contributed or deserves credit, it just does a best effort attempt at maintaining existing attribution we inherited and automating it going forward. The automation might expose existing problems - they should be fixed by adding or removing exceptions in the script.
author Mads Kiilerich <mads@kiilerich.com>
date Mon, 03 Jul 2017 00:54:36 +0200
parents
children 665dfa112f2c d06c0566cb23
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6777
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
1 #!/usr/bin/env python2
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
3
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
4 """
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
5 Kallithea script for maintaining contributor lists from version control
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
6 history.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
7
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
8 This script and the data in it is a best effort attempt at reverse engineering
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
9 previous attributions and correlate that with version control history while
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
10 preserving all existing copyright statements and attribution. This script is
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
11 processing and summarizing information found elsewhere - it is not by itself
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
12 making any claims. Comments in the script are an attempt at reverse engineering
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
13 possible explanations - they are not showing any intent or confirming it is
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
14 correct.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
15
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
16 Three files are generated / modified by this script:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
17
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
18 kallithea/templates/about.html claims to show copyright holders, and the GPL
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
19 license requires such existing "legal notices" to be preserved. We also try to
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
20 keep it updated with copyright holders, but do not claim it is a correct list.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
21
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
22 CONTRIBUTORS has the purpose of giving credit where credit is due and list all
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
23 the contributor names in the source.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
24
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
25 kallithea/templates/base/base.html contains the copyright years in the page
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
26 footer.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
27
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
28 Both make a best effort of listing all copyright holders, but revision control
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
29 history might be a better and more definitive source.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
30
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
31 Contributors are sorted "fairly" by copyright year and amount of
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
32 contribution.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
33
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
34 New contributors are listed, without considering if the contribution contains
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
35 copyrightable work.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
36
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
37 When the copyright might belong to a different legal entity than the
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
38 contributor, the legal entity is given credit too.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
39 """
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
40
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
41
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
42 # Some committers are so wrong that it doesn't point at any contributor:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
43 total_ignore = set()
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
44 total_ignore.add('*** failed to import extension hggit: No module named hggit')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
45 total_ignore.add('<>')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
46
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
47 # Normalize some committer names where people have contributed under different
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
48 # names or email addresses:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
49 name_fixes = {}
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
50 name_fixes['Andrew Shadura'] = "Andrew Shadura <andrew@shadura.me>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
51 name_fixes['aparkar'] = "Aparkar <aparkar@icloud.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
52 name_fixes['Aras Pranckevicius'] = "Aras Pranckevičius <aras@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
53 name_fixes['Augosto Hermann'] = "Augusto Herrmann <augusto.herrmann@planejamento.gov.br>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
54 name_fixes['"Bradley M. Kuhn" <bkuhn@ebb.org>'] = "Bradley M. Kuhn <bkuhn@sfconservancy.org>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
55 name_fixes['dmitri.kuznetsov'] = "Dmitri Kuznetsov"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
56 name_fixes['Dmitri Kuznetsov'] = "Dmitri Kuznetsov"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
57 name_fixes['domruf'] = "Dominik Ruf <dominikruf@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
58 name_fixes['Ingo von borstel'] = "Ingo von Borstel <kallithea@planetmaker.de>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
59 name_fixes['Jan Heylen'] = "Jan Heylen <heyleke@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
60 name_fixes['Jason F. Harris'] = "Jason Harris <jason@jasonfharris.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
61 name_fixes['Jelmer Vernooij'] = "Jelmer Vernooij <jelmer@samba.org>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
62 name_fixes['jfh <jason@jasonfharris.com>'] = "Jason Harris <jason@jasonfharris.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
63 name_fixes['Leonardo Carneiro<leonardo@unity3d.com>'] = "Leonardo Carneiro <leonardo@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
64 name_fixes['leonardo'] = "Leonardo Carneiro <leonardo@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
65 name_fixes['Leonardo <leo@unity3d.com>'] = "Leonardo Carneiro <leonardo@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
66 name_fixes['Les Peabody'] = "Les Peabody <lpeabody@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
67 name_fixes['"Lorenzo M. Catucci" <lorenzo@sancho.ccd.uniroma2.it>'] = "Lorenzo M. Catucci <lorenzo@sancho.ccd.uniroma2.it>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
68 name_fixes['Lukasz Balcerzak'] = "Łukasz Balcerzak <lukaszbalcerzak@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
69 name_fixes['mao <mao@lins.fju.edu.tw>'] = "Ching-Chen Mao <mao@lins.fju.edu.tw>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
70 name_fixes['marcink'] = "Marcin Kuźmiński <marcin@python-works.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
71 name_fixes['Marcin Kuzminski'] = "Marcin Kuźmiński <marcin@python-works.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
72 name_fixes['nansenat16@null.tw'] = "nansenat16 <nansenat16@null.tw>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
73 name_fixes['Peter Vitt'] = "Peter Vitt <petervitt@web.de>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
74 name_fixes['philip.j@hostdime.com'] = "Philip Jameson <philip.j@hostdime.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
75 name_fixes['Søren Løvborg'] = "Søren Løvborg <sorenl@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
76 name_fixes['Thomas De Schampheleire'] = "Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
77 name_fixes['Weblate'] = "<>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
78 name_fixes['xpol'] = "xpol <xpolife@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
79
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
80
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
81 # Some committer email address domains that indicate that another entity might
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
82 # hold some copyright too:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
83 domain_extra = {}
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
84 domain_extra['unity3d.com'] = "Unity Technologies"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
85 domain_extra['rhodecode.com'] = "RhodeCode GmbH"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
86
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
87 # Repository history show some old contributions that traditionally hasn't been
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
88 # listed in about.html - preserve that:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
89 no_about = set(total_ignore)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
90 # The following contributors were traditionally not listed in about.html and it
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
91 # seems unclear if the copyright is personal or belongs to a company.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
92 no_about.add(('Thayne Harbaugh <thayne@fusionio.com>', '2011'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
93 no_about.add(('Dies Koper <diesk@fast.au.fujitsu.com>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
94 no_about.add(('Erwin Kroon <e.kroon@smartmetersolutions.nl>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
95 no_about.add(('Vincent Caron <vcaron@bearstech.com>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
96 # These contributors' contributions might be too small to be copyrightable:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
97 no_about.add(('philip.j@hostdime.com', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
98 no_about.add(('Stefan Engel <mail@engel-stefan.de>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
99 no_about.add(('Ton Plomp <tcplomp@gmail.com>', '2013'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
100 # Was reworked and contributed later and shadowed by other contributions:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
101 no_about.add(('Sean Farley <sean.michael.farley@gmail.com>', '2013'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
102
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
103 # Preserve contributors listed in about.html but not appearing in repository
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
104 # history:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
105 other_about = [
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
106 ("2011", "Aparkar <aparkar@icloud.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
107 ("2010", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
108 ("2011", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
109 ("2012", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
110 ("2013", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
111 ]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
112
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
113 # Preserve contributors listed in CONTRIBUTORS but not appearing in repository
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
114 # history:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
115 other_contributors = [
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
116 ("", "Andrew Kesterson <andrew@aklabs.net>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
117 ("", "cejones"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
118 ("", "David A. Sjøen <david.sjoen@westcon.no>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
119 ("", "James Rhodes <jrhodes@redpointsoftware.com.au>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
120 ("", "Jonas Oberschweiber <jonas.oberschweiber@d-velop.de>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
121 ("", "larikale"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
122 ("", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
123 ("", "Sebastian Kreutzberger <sebastian@rhodecode.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
124 ("", "Steve Romanow <slestak989@gmail.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
125 ("", "SteveCohen"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
126 ("", "Thomas <thomas@rhodecode.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
127 ("", "Thomas Waldmann <tw-public@gmx.de>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
128 ]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
129
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
130
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
131 import os
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
132 import re
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
133 from collections import defaultdict
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
134
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
135
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
136 def sortkey(x):
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
137 """Return key for sorting contributors "fairly":
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
138 * latest contribution
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
139 * first contribution
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
140 * number of contribution years
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
141 * name (with some unicode normalization)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
142 The entries must be 2-tuples of a list of string years and the unicode name"""
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
143 return (x[0] and -int(x[0][-1]),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
144 x[0] and int(x[0][0]),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
145 -len(x[0]),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
146 x[1].decode('utf8').lower().replace(u'\xe9', u'e').replace(u'\u0142', u'l')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
147 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
148
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
149
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
150 def nice_years(l, dash='-', join=' '):
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
151 """Convert a list of years into brief range like '1900-1901, 1921'."""
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
152 if not l:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
153 return ''
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
154 start = end = int(l[0])
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
155 ranges = []
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
156 for year in l[1:] + [0]:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
157 year = int(year)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
158 if year == end + 1:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
159 end = year
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
160 continue
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
161 if start == end:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
162 ranges.append('%s' % start)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
163 else:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
164 ranges.append('%s%s%s' % (start, dash, end))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
165 start = end = year
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
166 assert start == 0 and end == 0, (start, end)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
167 return join.join(ranges)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
168
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
169
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
170 def insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
171 filename,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
172 all_entries,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
173 no_entries,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
174 domain_extra,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
175 split_re,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
176 normalize_name,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
177 format_f):
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
178 """Update file with contributor information.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
179 all_entries: list of tuples with year and name
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
180 no_entries: set of names or name and year tuples to ignore
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
181 domain_extra: map domain name to extra credit name
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
182 split_re: regexp matching the part of file to rewrite
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
183 normalize_name: function to normalize names for grouping and display
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
184 format_f: function formatting year list and name to a string
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
185 """
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
186 name_years = defaultdict(set)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
187
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
188 for year, name in all_entries:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
189 if name in no_entries or (name, year) in no_entries:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
190 continue
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
191 domain = name.split('@', 1)[-1].rstrip('>')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
192 if domain in domain_extra:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
193 name_years[domain_extra[domain]].add(year)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
194 name_years[normalize_name(name)].add(year)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
195
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
196 l = [(list(sorted(year for year in years if year)), name)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
197 for name, years in name_years.items()]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
198 l.sort(key=sortkey)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
199
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
200 with file(filename) as f:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
201 pre, post = re.split(split_re, f.read())
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
202
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
203 with file(filename, 'w') as f:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
204 f.write(pre +
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
205 ''.join(format_f(years, name) for years, name in l) +
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
206 post)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
207
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
208
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
209 def main():
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
210 repo_entries = [
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
211 (year, name_fixes.get(name) or name_fixes.get(name.rsplit('<', 1)[0].strip()) or name)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
212 for year, name in
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
213 (line.strip().split(' ', 1)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
214 for line in os.popen("""hg log -r '::.' -T '{date(date,"%Y")} {author}\n'""").readlines())
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
215 ]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
216
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
217 insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
218 filename='kallithea/templates/about.html',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
219 all_entries=repo_entries + other_about,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
220 no_entries=no_about,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
221 domain_extra=domain_extra,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
222 split_re=r'(?: <li>Copyright &copy; [^\n]*</li>\n)*',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
223 normalize_name=lambda name: name.split('<', 1)[0].strip(),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
224 format_f=lambda years, name: ' <li>Copyright &copy; %s, %s</li>\n' % (nice_years(years, '&ndash;', ', '), name),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
225 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
226
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
227 insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
228 filename='CONTRIBUTORS',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
229 all_entries=repo_entries + other_contributors,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
230 no_entries=total_ignore,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
231 domain_extra=domain_extra,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
232 split_re=r'(?: [^\n]*\n)*',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
233 normalize_name=lambda name: name,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
234 format_f=lambda years, name: (' %s%s%s\n' % (name, ' ' if years else '', nice_years(years))),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
235 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
236
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
237 insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
238 filename='kallithea/templates/base/base.html',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
239 all_entries=repo_entries,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
240 no_entries=total_ignore,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
241 domain_extra={},
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
242 split_re=r'(?<=&copy;) .* (?=by various authors)',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
243 normalize_name=lambda name: '',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
244 format_f=lambda years, name: ' ' + nice_years(years, '&ndash;', ', ') + ' ',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
245 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
246
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
247
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
248 if __name__ == '__main__':
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
249 main()
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
250
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
251
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
252 # To list new contributors since last tagging:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
253 # { hg log -r '::tagged()' -T ' {author}\n {author}\n'; hg log -r '::.' -T ' {author}\n' | sort | uniq; } | sort | uniq -u