-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathREADME.html
440 lines (412 loc) · 16.2 KB
/
README.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.11: http://docutils.sourceforge.net/" />
<title>Erlang + Erlport + Python templates</title>
<meta name="author" content="Dave Kuhlman" />
<meta name="date" content="April 10, 2013" />
<style type="text/css">
/* css */
body {
font: 90% 'Lucida Grande', Verdana, Geneva, Lucida, Arial, Helvetica, sans-serif;
background: #ffffff;
color: black;
margin: 2em;
padding: 2em;
}
a[href] {
color: #436976;
background-color: transparent;
}
a.toc-backref {
text-decoration: none;
}
h1 a[href] {
text-decoration: none;
color: #fcb100;
background-color: transparent;
}
a.strong {
font-weight: bold;
}
img {
margin: 0;
border: 0;
}
p {
margin: 0.5em 0 1em 0;
line-height: 1.5em;
}
p a {
text-decoration: underline;
}
p a:visited {
color: purple;
background-color: transparent;
}
p a:active {
color: red;
background-color: transparent;
}
a:hover {
text-decoration: none;
}
p img {
border: 0;
margin: 0;
}
h1, h2, h3, h4, h5, h6 {
color: #003a6b;
background-color: transparent;
font: 100% 'Lucida Grande', Verdana, Geneva, Lucida, Arial, Helvetica, sans-serif;
margin: 0;
padding-top: 0.5em;
}
h1 {
font-size: 160%;
margin-bottom: 0.5em;
border-bottom: 1px solid #fcb100;
}
h2 {
font-size: 140%;
margin-bottom: 0.5em;
border-bottom: 1px solid #aaa;
}
h3 {
font-size: 130%;
margin-bottom: 0.5em;
text-decoration: underline;
}
h4 {
font-size: 110%;
font-weight: bold;
}
h5 {
font-size: 100%;
font-weight: bold;
}
h6 {
font-size: 80%;
font-weight: bold;
}
ul a, ol a {
text-decoration: underline;
}
dt {
font-weight: bold;
}
dt a {
text-decoration: none;
}
dd {
line-height: 1.5em;
margin-bottom: 1em;
}
legend {
background: #ffffff;
padding: 0.5em;
}
form {
margin: 0;
}
dl.form {
margin: 0;
padding: 1em;
}
dl.form dt {
width: 30%;
float: left;
margin: 0;
padding: 0 0.5em 0.5em 0;
text-align: right;
}
input {
font: 100% 'Lucida Grande', Verdana, Geneva, Lucida, Arial, Helvetica, sans-serif;
color: black;
background-color: white;
vertical-align: middle;
}
abbr, acronym, .explain {
color: black;
background-color: transparent;
}
q, blockquote {
}
code, pre {
font-family: monospace;
font-size: 1.2em;
display: block;
padding: 10px;
border: 1px solid #838183;
background-color: #eee;
color: #000;
overflow: auto;
margin: 0.5em 1em;
}
div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
margin: 2em ;
border: medium outset ;
padding: 1em }
div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
font-weight: bold ;
font-family: sans-serif }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
color: red ;
font-weight: bold ;
font-family: sans-serif }
tt.docutils {
background-color: #eeeeee;
}
</style>
</head>
<body>
<div class="document" id="erlang-erlport-python-templates">
<h1 class="title">Erlang + Erlport + Python templates</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Dave Kuhlman</td></tr>
<tr><th class="docinfo-name">Address:</th>
<td><pre class="address">
<a class="first reference external" href="mailto:dkuhlman@rexx.com">dkuhlman@rexx.com</a>
<a class="last reference external" href="http://www.rexx.com/~dkuhlman">http://www.rexx.com/~dkuhlman</a>
</pre>
</td></tr>
<tr><th class="docinfo-name">Revision:</th>
<td>1.0a</td></tr>
<tr><th class="docinfo-name">Date:</th>
<td>April 10, 2013</td></tr>
</tbody>
</table>
<!-- vim: set ft=rst: -->
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field"><th class="field-name">copyright:</th><td class="field-body">Copyright (c) 2013 Dave Kuhlman. This documentation
and the software it describes is covered by The MIT License:
<a class="reference external" href="http://www.opensource.org/licenses/mit-license.php">http://www.opensource.org/licenses/mit-license.php</a>.</td>
</tr>
<tr class="field"><th class="field-name">abstract:</th><td class="field-body">This document describes a set of templates that can be
used to ease the task of creating Erlang/erlport/Python modules.</td>
</tr>
</tbody>
</table>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="auto-toc simple">
<li><a class="reference internal" href="#description" id="id1">1 Description</a></li>
<li><a class="reference internal" href="#instructions" id="id2">2 Instructions</a><ul class="auto-toc">
<li><a class="reference internal" href="#example-code" id="id3">2.1 Example code</a></li>
<li><a class="reference internal" href="#requests" id="id4">2.2 Requests</a></li>
</ul>
</li>
<li><a class="reference internal" href="#the-specific-templates" id="id5">3 The specific templates</a><ul class="auto-toc">
<li><a class="reference internal" href="#requestor01-and-client01" id="id6">3.1 requestor01 and client01</a></li>
<li><a class="reference internal" href="#requestor02-and-client02" id="id7">3.2 requestor02 and client02</a></li>
<li><a class="reference internal" href="#requestor03-and-client03" id="id8">3.3 requestor03 and client03</a></li>
<li><a class="reference internal" href="#requestor04-and-client04" id="id9">3.4 requestor04 and client04</a></li>
<li><a class="reference internal" href="#requestor05-and-client05" id="id10">3.5 requestor05 and client05</a></li>
<li><a class="reference internal" href="#requestor06-and-client06-an-erlang-behavior" id="id11">3.6 requestor06 and client06 -- An Erlang behavior</a></li>
</ul>
</li>
<li><a class="reference internal" href="#possible-needed-additions-and-extensions" id="id12">4 Possible, needed additions and extensions</a></li>
</ul>
</div>
<div class="section" id="description">
<h1><a class="toc-backref" href="#id1">1 Description</a></h1>
<p>This package contains a set of templates that can be used to
interface Erlang to Python using Erlport.</p>
<p>Using one pair of these files (requestor + client), you should be
able to quickly create a harness for the Python processing that you
want to request from within an Erlang module.</p>
<p>For information about Erlport, see:
<a class="reference external" href="https://github.com/hdima/erlport.git">https://github.com/hdima/erlport.git</a></p>
</div>
<div class="section" id="instructions">
<h1><a class="toc-backref" href="#id2">2 Instructions</a></h1>
<ol class="arabic">
<li><p class="first">Examine the requestors (<tt class="docutils literal">requestorN.erl</tt>). Choose the style of
interface that suits your needs.</p>
</li>
<li><p class="first">Copy and rename the requestor that your have chosen and the
corresponding client: client01.py for requestor01.py, client02,py
for requestor02.py, etc.</p>
</li>
<li><p class="first">Edit the requestor and the client:</p>
<ul class="simple">
<li>In the requestor, change the module name.</li>
<li>In both the requestor and client, change all variable names and
constants that are prefixed with "fix_".</li>
<li>In the client, add the Python processing that you wish to
perform in response to each different request. Add new
handlers or delete unneeded ones. The handler should return
whatever data structure that you want to use to pass data back
to the requestor, for example, you can use Python lists and
tuples, which will be converted into Erlang lists and tuples.
To return an Erlang atom, use <tt class="docutils literal"><span class="pre">erlport.Atom('my_atom_name')</span></tt>.</li>
<li>In the requestor, add the processing that you wish to perform
to the data returned by the client.</li>
</ul>
</li>
<li><p class="first">Compile the requestor for Erlang.</p>
</li>
<li><p class="first">Test your code. For example:</p>
<pre class="literal-block">
1> c(my_requestor).
2> P = my_requestor:start()
3> my_requestor:rpc(P, {get_value, "some_file.xml"}).
o
o
o
9> my_requestor:quit(P).
</pre>
</li>
</ol>
<div class="section" id="example-code">
<h2><a class="toc-backref" href="#id3">2.1 Example code</a></h2>
<p>For examples of demo applications built with the use of these
templates, see the README and sample code in the <tt class="docutils literal">examples</tt>
subdirectory.</p>
</div>
<div class="section" id="requests">
<h2><a class="toc-backref" href="#id4">2.2 Requests</a></h2>
<p>A word about the requests and their format. These templates assume
that we need a way to communicate several different kinds of
requests to the Python process. So, we send a tuple in which the
first item (an atom) specifies the kind of request and in which the
remaining arguments are arguments to the Python handler. So, for
example, if the request is:</p>
<pre class="literal-block">
{op_code_1, arg_1, arg_2, arg_3}
</pre>
<p>then the Python handler (method) should have the following header:</p>
<pre class="literal-block">
def handle_op_code_1(self, arg1, arg2, arg3):
</pre>
</div>
</div>
<div class="section" id="the-specific-templates">
<h1><a class="toc-backref" href="#id5">3 The specific templates</a></h1>
<div class="section" id="requestor01-and-client01">
<h2><a class="toc-backref" href="#id6">3.1 requestor01 and client01</a></h2>
<p>This template creates a new port and a new Python process for each
request. Remember that Python processes are "heavy weight": the are
operating system processes that show up in the Task Manager on MS
Windows and in the System Monitor on Linux.</p>
<p>Use this template if your requests to Python are few and where you
want those Python/OS processes to disappear between requests.</p>
</div>
<div class="section" id="requestor02-and-client02">
<h2><a class="toc-backref" href="#id7">3.2 requestor02 and client02</a></h2>
<p>This one creates opens a port (and creates the associated Python
process), then returns that port to the caller. You can then call
one of several functions in the module to make separate and
different requests using the same Python process. This template
reuse the same Python process so the use of that process is
synchronous. However, you can open more that one port and use them
concurrently. And, you can close the port, destroying the Python
process, when you are finished with it.</p>
<p>Use this template when you need to make a number of requests to your
Python module in sequence and do <em>not</em> want to re-create the Python
process for each request.</p>
</div>
<div class="section" id="requestor03-and-client03">
<h2><a class="toc-backref" href="#id8">3.3 requestor03 and client03</a></h2>
<p>This one is similar to requestor02/client02 in that the start/0
function creates and gives you a port/process that you can use
repeatedly (sequentially), then later destroy when you no longer
need it. However, the user interface uses the rpc request style.
(See Joe Armstrong, "Programming Erlang" for more on the <tt class="docutils literal">rpc</tt>
style of interface.) Using this template, you would interface with
your Python process by calling the <tt class="docutils literal">rpc</tt> function passing in the
port and an Erlang term that specifies the request.</p>
<p>This template might be particularly well suited in an application
where a sequence of tasks are specified in consistently structured
data objects.</p>
</div>
<div class="section" id="requestor04-and-client04">
<h2><a class="toc-backref" href="#id9">3.4 requestor04 and client04</a></h2>
<p>This template creates a set of Python processes of a fixed size. It
then runs the requests from a list of requests on those processes,
starting up a new task on an existing process when that process
completes its previous task.</p>
<p>Use this template when the processing on the Python side is likely
to be intensive and compute bound, but the processing on the Erlang
side (e.g. saving the results) is likely to be light weight.
Applications built with this template should be able to keep
multiple cores busy when most of the work is on the Python side.</p>
<p>Also consider using this template when the processing on the Erlang
side needs coordination, for example, when you need to merge results
from multiple processes on the Python side, when you need to
compare the results from multiple processes on the Python side
(e.g. to select an optimum of some kind, etc.</p>
</div>
<div class="section" id="requestor05-and-client05">
<h2><a class="toc-backref" href="#id10">3.5 requestor05 and client05</a></h2>
<p>This template creates a fixed size set of processes and runs
(distributes) tasks across those processes. Each of these processes
is composed of an Erlang process and an associated Erlport Python
(OS) process.</p>
<p>Use this template when the processing on both the Erlang side and
the Python side of the port is likely to be intensive (or maybe even
when the processing on the Erlang side will be intensive and the
processing on the Python side is not). Since this template treats
the Erlang code and the Python code as part of the same (Erlang)
process and since it parallelizes those process pairs, applications
built with this template will keep multiple cores busy even when
some (or even most) of the heavy processing is on the Erlang side of
that pair.</p>
<p>Remember that, since the Erlang processing is being done in separate
processes, these processes can communicate with each other or
with the supervisor process that started them only through messages.</p>
</div>
<div class="section" id="requestor06-and-client06-an-erlang-behavior">
<h2><a class="toc-backref" href="#id11">3.6 requestor06 and client06 -- An Erlang behavior</a></h2>
<p>This template creates a fixed size set of processes and keeps them
in a "pool" of processes. Whenever a task is requested, a process
is taken out of the pool and given that task to process. When the
Python process is complete (specifically, when it returns the atom
"finisted", the process is put back in the pool and made available.</p>
<p>This "template" is implemented with an Erlang behavior. This
behavior enables you to implement the erang process controller with
a very small number of lines of code.</p>
<p>An example of an Erlang process controller that uses this behavior
is provided by <tt class="docutils literal">requestor06test.erl</tt>.</p>
<p>You can try out this template/behavior by running <tt class="docutils literal">test06_run</tt>,
for example:</p>
<pre class="literal-block">
$ ./test06_run 3 5
</pre>
<p>By default, this example uses the file <tt class="docutils literal">client01.xml</tt> for input.</p>
</div>
</div>
<div class="section" id="possible-needed-additions-and-extensions">
<h1><a class="toc-backref" href="#id12">4 Possible, needed additions and extensions</a></h1>
<ul class="simple">
<li>Support for Python processes where the processing on the Python
side needs to return results multiple times or needs to return its
results in a sequence of chunks (a sequence of messages), rather
than in a single chunk (message).</li>
<li>Enable user to feed tasks using a function (generator?) rather
than a list, so that tasks can be generated dynamically, perhaps
even in response to previous computed results.</li>
</ul>
</div>
</div>
<div class="footer">
<hr class="footer" />
<a class="reference external" href="README.rst">View document source</a>.
Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
</div>
</body>
</html>