Mercurial > kallithea
annotate rhodecode/lib/subprocessio.py @ 3129:23ec6e88b234 beta
Merged translation from pullrequest #92
author | Marcin Kuzminski <marcin@python-works.com> |
---|---|
date | Wed, 02 Jan 2013 00:32:36 +0100 |
parents | 4b7ad342e53b |
children | 64c194492aad |
rev | line source |
---|---|
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
1 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
2 Module provides a class allowing to wrap communication over subprocess.Popen |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
3 input, output, error streams into a meaningfull, non-blocking, concurrent |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
4 stream processor exposing the output data as an iterator fitting to be a |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
5 return value passed by a WSGI applicaiton to a WSGI server per PEP 3333. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
6 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
7 Copyright (c) 2011 Daniel Dotsenko <dotsa@hotmail.com> |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
8 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
9 This file is part of git_http_backend.py Project. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
10 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
11 git_http_backend.py Project is free software: you can redistribute it and/or |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
12 modify it under the terms of the GNU Lesser General Public License as |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
13 published by the Free Software Foundation, either version 2.1 of the License, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
14 or (at your option) any later version. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
15 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
16 git_http_backend.py Project is distributed in the hope that it will be useful, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
17 but WITHOUT ANY WARRANTY; without even the implied warranty of |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
18 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
19 GNU Lesser General Public License for more details. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
20 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
21 You should have received a copy of the GNU Lesser General Public License |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
22 along with git_http_backend.py Project. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
23 If not, see <http://www.gnu.org/licenses/>. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
24 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
25 import os |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
26 import subprocess |
2826
4b7ad342e53b
python2.5 fixes !
Marcin Kuzminski <marcin@python-works.com>
parents:
2790
diff
changeset
|
27 from rhodecode.lib.compat import deque, Event, Thread, _bytes, _bytearray |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
28 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
29 |
2790
3c0ae44557c4
more py2.5 compatibility patches
Marcin Kuzminski <marcin@python-works.com>
parents:
2760
diff
changeset
|
30 class StreamFeeder(Thread): |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
31 """ |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
32 Normal writing into pipe-like is blocking once the buffer is filled. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
33 This thread allows a thread to seep data from a file-like into a pipe |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
34 without blocking the main thread. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
35 We close inpipe once the end of the source stream is reached. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
36 """ |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
37 def __init__(self, source): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
38 super(StreamFeeder, self).__init__() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
39 self.daemon = True |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
40 filelike = False |
2790
3c0ae44557c4
more py2.5 compatibility patches
Marcin Kuzminski <marcin@python-works.com>
parents:
2760
diff
changeset
|
41 self.bytes = _bytes() |
2826
4b7ad342e53b
python2.5 fixes !
Marcin Kuzminski <marcin@python-works.com>
parents:
2790
diff
changeset
|
42 if type(source) in (type(''), _bytes, _bytearray): # string-like |
2790
3c0ae44557c4
more py2.5 compatibility patches
Marcin Kuzminski <marcin@python-works.com>
parents:
2760
diff
changeset
|
43 self.bytes = _bytes(source) |
2564
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
44 else: # can be either file pointer or file-like |
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
45 if type(source) in (int, long): # file pointer it is |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
46 ## converting file descriptor (int) stdin into file-like |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
47 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
48 source = os.fdopen(source, 'rb', 16384) |
2564
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
49 except Exception: |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
50 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
51 # let's see if source is file-like by now |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
52 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
53 filelike = source.read |
2564
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
54 except Exception: |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
55 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
56 if not filelike and not self.bytes: |
2564
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
57 raise TypeError("StreamFeeder's source object must be a readable " |
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
58 "file-like, a file descriptor, or a string-like.") |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
59 self.source = source |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
60 self.readiface, self.writeiface = os.pipe() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
61 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
62 def run(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
63 t = self.writeiface |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
64 if self.bytes: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
65 os.write(t, self.bytes) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
66 else: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
67 s = self.source |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
68 b = s.read(4096) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
69 while b: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
70 os.write(t, b) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
71 b = s.read(4096) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
72 os.close(t) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
73 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
74 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
75 def output(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
76 return self.readiface |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
77 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
78 |
2790
3c0ae44557c4
more py2.5 compatibility patches
Marcin Kuzminski <marcin@python-works.com>
parents:
2760
diff
changeset
|
79 class InputStreamChunker(Thread): |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
80 def __init__(self, source, target, buffer_size, chunk_size): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
81 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
82 super(InputStreamChunker, self).__init__() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
83 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
84 self.daemon = True # die die die. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
85 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
86 self.source = source |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
87 self.target = target |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
88 self.chunk_count_max = int(buffer_size / chunk_size) + 1 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
89 self.chunk_size = chunk_size |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
90 |
2730
7949bc80b3b1
more py25 compat fixes
Marcin Kuzminski <marcin@python-works.com>
parents:
2729
diff
changeset
|
91 self.data_added = Event() |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
92 self.data_added.clear() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
93 |
2730
7949bc80b3b1
more py25 compat fixes
Marcin Kuzminski <marcin@python-works.com>
parents:
2729
diff
changeset
|
94 self.keep_reading = Event() |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
95 self.keep_reading.set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
96 |
2730
7949bc80b3b1
more py25 compat fixes
Marcin Kuzminski <marcin@python-works.com>
parents:
2729
diff
changeset
|
97 self.EOF = Event() |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
98 self.EOF.clear() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
99 |
2730
7949bc80b3b1
more py25 compat fixes
Marcin Kuzminski <marcin@python-works.com>
parents:
2729
diff
changeset
|
100 self.go = Event() |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
101 self.go.set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
102 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
103 def stop(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
104 self.go.clear() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
105 self.EOF.set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
106 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
107 # this is not proper, but is done to force the reader thread let |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
108 # go of the input because, if successful, .close() will send EOF |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
109 # down the pipe. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
110 self.source.close() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
111 except: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
112 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
113 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
114 def run(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
115 s = self.source |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
116 t = self.target |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
117 cs = self.chunk_size |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
118 ccm = self.chunk_count_max |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
119 kr = self.keep_reading |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
120 da = self.data_added |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
121 go = self.go |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
122 b = s.read(cs) |
2790
3c0ae44557c4
more py2.5 compatibility patches
Marcin Kuzminski <marcin@python-works.com>
parents:
2760
diff
changeset
|
123 |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
124 while b and go.is_set(): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
125 if len(t) > ccm: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
126 kr.clear() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
127 kr.wait(2) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
128 # # this only works on 2.7.x and up |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
129 # if not kr.wait(10): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
130 # raise Exception("Timed out while waiting for input to be read.") |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
131 # instead we'll use this |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
132 if len(t) > ccm + 3: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
133 raise IOError("Timed out while waiting for input from subprocess.") |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
134 t.append(b) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
135 da.set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
136 b = s.read(cs) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
137 self.EOF.set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
138 da.set() # for cases when done but there was no input. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
139 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
140 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
141 class BufferedGenerator(): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
142 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
143 Class behaves as a non-blocking, buffered pipe reader. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
144 Reads chunks of data (through a thread) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
145 from a blocking pipe, and attaches these to an array (Deque) of chunks. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
146 Reading is halted in the thread when max chunks is internally buffered. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
147 The .next() may operate in blocking or non-blocking fashion by yielding |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
148 '' if no data is ready |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
149 to be sent or by not returning until there is some data to send |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
150 When we get EOF from underlying source pipe we raise the marker to raise |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
151 StopIteration after the last chunk of data is yielded. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
152 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
153 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
154 def __init__(self, source, buffer_size=65536, chunk_size=4096, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
155 starting_values=[], bottomless=False): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
156 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
157 if bottomless: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
158 maxlen = int(buffer_size / chunk_size) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
159 else: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
160 maxlen = None |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
161 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
162 self.data = deque(starting_values, maxlen) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
163 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
164 self.worker = InputStreamChunker(source, self.data, buffer_size, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
165 chunk_size) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
166 if starting_values: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
167 self.worker.data_added.set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
168 self.worker.start() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
169 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
170 #################### |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
171 # Generator's methods |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
172 #################### |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
173 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
174 def __iter__(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
175 return self |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
176 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
177 def next(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
178 while not len(self.data) and not self.worker.EOF.is_set(): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
179 self.worker.data_added.clear() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
180 self.worker.data_added.wait(0.2) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
181 if len(self.data): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
182 self.worker.keep_reading.set() |
2790
3c0ae44557c4
more py2.5 compatibility patches
Marcin Kuzminski <marcin@python-works.com>
parents:
2760
diff
changeset
|
183 return _bytes(self.data.popleft()) |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
184 elif self.worker.EOF.is_set(): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
185 raise StopIteration |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
186 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
187 def throw(self, type, value=None, traceback=None): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
188 if not self.worker.EOF.is_set(): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
189 raise type(value) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
190 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
191 def start(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
192 self.worker.start() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
193 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
194 def stop(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
195 self.worker.stop() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
196 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
197 def close(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
198 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
199 self.worker.stop() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
200 self.throw(GeneratorExit) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
201 except (GeneratorExit, StopIteration): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
202 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
203 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
204 def __del__(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
205 self.close() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
206 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
207 #################### |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
208 # Threaded reader's infrastructure. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
209 #################### |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
210 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
211 def input(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
212 return self.worker.w |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
213 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
214 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
215 def data_added_event(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
216 return self.worker.data_added |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
217 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
218 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
219 def data_added(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
220 return self.worker.data_added.is_set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
221 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
222 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
223 def reading_paused(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
224 return not self.worker.keep_reading.is_set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
225 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
226 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
227 def done_reading_event(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
228 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
229 Done_reding does not mean that the iterator's buffer is empty. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
230 Iterator might have done reading from underlying source, but the read |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
231 chunks might still be available for serving through .next() method. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
232 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
233 @return An Event class instance. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
234 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
235 return self.worker.EOF |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
236 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
237 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
238 def done_reading(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
239 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
240 Done_reding does not mean that the iterator's buffer is empty. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
241 Iterator might have done reading from underlying source, but the read |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
242 chunks might still be available for serving through .next() method. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
243 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
244 @return An Bool value. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
245 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
246 return self.worker.EOF.is_set() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
247 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
248 @property |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
249 def length(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
250 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
251 returns int. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
252 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
253 This is the lenght of the que of chunks, not the length of |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
254 the combined contents in those chunks. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
255 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
256 __len__() cannot be meaningfully implemented because this |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
257 reader is just flying throuh a bottomless pit content and |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
258 can only know the lenght of what it already saw. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
259 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
260 If __len__() on WSGI server per PEP 3333 returns a value, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
261 the responce's length will be set to that. In order not to |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
262 confuse WSGI PEP3333 servers, we will not implement __len__ |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
263 at all. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
264 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
265 return len(self.data) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
266 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
267 def prepend(self, x): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
268 self.data.appendleft(x) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
269 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
270 def append(self, x): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
271 self.data.append(x) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
272 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
273 def extend(self, o): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
274 self.data.extend(o) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
275 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
276 def __getitem__(self, i): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
277 return self.data[i] |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
278 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
279 |
2399
a8635cdab3c0
Add optional parameters to subprocessio that allow passing params to Popen
Marcin Kuzminski <marcin@python-works.com>
parents:
2382
diff
changeset
|
280 class SubprocessIOChunker(object): |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
281 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
282 Processor class wrapping handling of subprocess IO. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
283 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
284 In a way, this is a "communicate()" replacement with a twist. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
285 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
286 - We are multithreaded. Writing in and reading out, err are all sep threads. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
287 - We support concurrent (in and out) stream processing. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
288 - The output is not a stream. It's a queue of read string (bytes, not unicode) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
289 chunks. The object behaves as an iterable. You can "for chunk in obj:" us. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
290 - We are non-blocking in more respects than communicate() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
291 (reading from subprocess out pauses when internal buffer is full, but |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
292 does not block the parent calling code. On the flip side, reading from |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
293 slow-yielding subprocess may block the iteration until data shows up. This |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
294 does not block the parallel inpipe reading occurring parallel thread.) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
295 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
296 The purpose of the object is to allow us to wrap subprocess interactions into |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
297 and interable that can be passed to a WSGI server as the application's return |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
298 value. Because of stream-processing-ability, WSGI does not have to read ALL |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
299 of the subprocess's output and buffer it, before handing it to WSGI server for |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
300 HTTP response. Instead, the class initializer reads just a bit of the stream |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
301 to figure out if error ocurred or likely to occur and if not, just hands the |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
302 further iteration over subprocess output to the server for completion of HTTP |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
303 response. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
304 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
305 The real or perceived subprocess error is trapped and raised as one of |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
306 EnvironmentError family of exceptions |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
307 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
308 Example usage: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
309 # try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
310 # answer = SubprocessIOChunker( |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
311 # cmd, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
312 # input, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
313 # buffer_size = 65536, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
314 # chunk_size = 4096 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
315 # ) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
316 # except (EnvironmentError) as e: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
317 # print str(e) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
318 # raise e |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
319 # |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
320 # return answer |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
321 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
322 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
323 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
324 def __init__(self, cmd, inputstream=None, buffer_size=65536, |
2399
a8635cdab3c0
Add optional parameters to subprocessio that allow passing params to Popen
Marcin Kuzminski <marcin@python-works.com>
parents:
2382
diff
changeset
|
325 chunk_size=4096, starting_values=[], **kwargs): |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
326 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
327 Initializes SubprocessIOChunker |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
328 |
2564
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
329 :param cmd: A Subprocess.Popen style "cmd". Can be string or array of strings |
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
330 :param inputstream: (Default: None) A file-like, string, or file pointer. |
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
331 :param buffer_size: (Default: 65536) A size of total buffer per stream in bytes. |
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
332 :param chunk_size: (Default: 4096) A max size of a chunk. Actual chunk may be smaller. |
9a99b574bb48
py25 compatibility for subprocession module
Marcin Kuzminski <marcin@python-works.com>
parents:
2399
diff
changeset
|
333 :param starting_values: (Default: []) An array of strings to put in front of output que. |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
334 ''' |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
335 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
336 if inputstream: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
337 input_streamer = StreamFeeder(inputstream) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
338 input_streamer.start() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
339 inputstream = input_streamer.output |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
340 |
2676
1f4d4b8d72f5
switched git_command to subprocession for non-blocking Popen.
Marcin Kuzminski <marcin@python-works.com>
parents:
2572
diff
changeset
|
341 if isinstance(cmd, (list, tuple)): |
1f4d4b8d72f5
switched git_command to subprocession for non-blocking Popen.
Marcin Kuzminski <marcin@python-works.com>
parents:
2572
diff
changeset
|
342 cmd = ' '.join(cmd) |
1f4d4b8d72f5
switched git_command to subprocession for non-blocking Popen.
Marcin Kuzminski <marcin@python-works.com>
parents:
2572
diff
changeset
|
343 |
2760
fd5f2b217488
get stderr also for git commands, pass in shell = False
Marcin Kuzminski <marcin@python-works.com>
parents:
2730
diff
changeset
|
344 _shell = kwargs.get('shell') or True |
fd5f2b217488
get stderr also for git commands, pass in shell = False
Marcin Kuzminski <marcin@python-works.com>
parents:
2730
diff
changeset
|
345 kwargs['shell'] = _shell |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
346 _p = subprocess.Popen(cmd, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
347 bufsize=-1, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
348 stdin=inputstream, |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
349 stdout=subprocess.PIPE, |
2399
a8635cdab3c0
Add optional parameters to subprocessio that allow passing params to Popen
Marcin Kuzminski <marcin@python-works.com>
parents:
2382
diff
changeset
|
350 stderr=subprocess.PIPE, |
a8635cdab3c0
Add optional parameters to subprocessio that allow passing params to Popen
Marcin Kuzminski <marcin@python-works.com>
parents:
2382
diff
changeset
|
351 **kwargs |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
352 ) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
353 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
354 bg_out = BufferedGenerator(_p.stdout, buffer_size, chunk_size, starting_values) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
355 bg_err = BufferedGenerator(_p.stderr, 16000, 1, bottomless=True) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
356 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
357 while not bg_out.done_reading and not bg_out.reading_paused and not bg_err.length: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
358 # doing this until we reach either end of file, or end of buffer. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
359 bg_out.data_added_event.wait(1) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
360 bg_out.data_added_event.clear() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
361 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
362 # at this point it's still ambiguous if we are done reading or just full buffer. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
363 # Either way, if error (returned by ended process, or implied based on |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
364 # presence of stuff in stderr output) we error out. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
365 # Else, we are happy. |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
366 _returncode = _p.poll() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
367 if _returncode or (_returncode == None and bg_err.length): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
368 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
369 _p.terminate() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
370 except: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
371 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
372 bg_out.stop() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
373 bg_err.stop() |
2676
1f4d4b8d72f5
switched git_command to subprocession for non-blocking Popen.
Marcin Kuzminski <marcin@python-works.com>
parents:
2572
diff
changeset
|
374 err = '%s' % ''.join(bg_err) |
1f4d4b8d72f5
switched git_command to subprocession for non-blocking Popen.
Marcin Kuzminski <marcin@python-works.com>
parents:
2572
diff
changeset
|
375 raise EnvironmentError("Subprocess exited due to an error:\n" + err) |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
376 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
377 self.process = _p |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
378 self.output = bg_out |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
379 self.error = bg_err |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
380 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
381 def __iter__(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
382 return self |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
383 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
384 def next(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
385 if self.process.poll(): |
2676
1f4d4b8d72f5
switched git_command to subprocession for non-blocking Popen.
Marcin Kuzminski <marcin@python-works.com>
parents:
2572
diff
changeset
|
386 err = '%s' % ''.join(self.error) |
2572
71fc1c98e02a
dont format errors to string in subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
2565
diff
changeset
|
387 raise EnvironmentError("Subprocess exited due to an error:\n" + err) |
2382
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
388 return self.output.next() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
389 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
390 def throw(self, type, value=None, traceback=None): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
391 if self.output.length or not self.output.done_reading: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
392 raise type(value) |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
393 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
394 def close(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
395 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
396 self.process.terminate() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
397 except: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
398 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
399 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
400 self.output.close() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
401 except: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
402 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
403 try: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
404 self.error.close() |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
405 except: |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
406 pass |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
407 |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
408 def __del__(self): |
034e4fe1ebb2
changed dulwich git interface to gitweb + subprocessio
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
409 self.close() |