diff options
| author | Hans-Christoph Steiner <eighthave@users.sourceforge.net> | 2005-12-16 00:53:00 +0000 |
|---|---|---|
| committer | Hans-Christoph Steiner <eighthave@users.sourceforge.net> | 2005-12-16 00:53:00 +0000 |
| commit | e28a07fba67af0af818dda6afa4cf67c09700816 (patch) | |
| tree | b0e8f2c7817bd72b8a6ce8353b1743c764a3a83a /doc/misc | |
| parent | c05865f60bf15e962b6b30eceb8fa81358f1eaa2 (diff) | |
checking in pdp 0.12.4 from http://zwizwa.fartit.com/pd/pdp/pdp-0.12.4.tar.gz
svn path=/trunk/externals/pdp/; revision=4231
Diffstat (limited to 'doc/misc')
| -rw-r--r-- | doc/misc/devdoc.html | 167 | ||||
| -rw-r--r-- | doc/misc/layers.txt | 222 | ||||
| -rw-r--r-- | doc/misc/overview.html | 102 | ||||
| -rw-r--r-- | doc/misc/todo.jme | 4 |
4 files changed, 495 insertions, 0 deletions
diff --git a/doc/misc/devdoc.html b/doc/misc/devdoc.html new file mode 100644 index 0000000..5ce1f83 --- /dev/null +++ b/doc/misc/devdoc.html @@ -0,0 +1,167 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <title>PDP Developer Documentation</title> + </head> + + <body> + <h1>PDP Developer Documentation</h1> + + <h2>Introduction</h2> + + <p>There is not yet much developer information, partly because pdp is not that big and since the goals are + not completely clear yet, a lot will probably change on the inside in the future. I believe it is + not too hard to figure out how it works, once you get started somewhere. This document is a minimalistic + attempt to provide that starting point. For full prototypes see the header files. I suggest you have a look at the pdp_base base class, and some simple + modules: pdp_add, pdp_noise and pdp_gain for examples. + + <h2> PDP architecture </h2> + <p> Architecture is a big word, but pdp is organized as modules. A packet pool module (a reuse pool memory manager), + a packet class, a processing queue module, a high level type conversion module, an image packet class, and some + low level modules for image type conversion, image resampling and all sorts of other image processing. Besides that + there are 2 extension libraries: pdp_scaf, a cellular automata extension and pdp_opengl, a 3d rendering extension. + These are separate because of portability issues. The different pdp_* externs in the main pdp library use the + core modules' functionality to minimize code duplication. I'm relatively happy with how it fits together, + but some things need to change for future plans. Most objects are written in the object oriented c style of pd. + To prevent namespace conflicts, (almost) all routines start with the pdp_ prefix. The second name is the name of the + object or module they belong to. The first argument is always a pointer to an object or an integer (for packets). + + + <h2> PD ties </h2> + <p> PDP is written as an extension for PD. One of the goals of pdp is to evolve to a separate library that can + be reused in other software. The architecture will be split into two parts. A pd-independent part (the packet classes, + the packet pool, the type conversion system and the forth system) and a part with pd specific stuff (the process queue and interfaces to the + pd system like the base classes and the pd communication protocol). In order to do this the packet class will probably + evolve to a proper object model, supporting run time attribute binding (inspired by the python object model). + + <p>There are some things that put a stamp on the current pdp design. Most importantly pd's processor object model and + communication protocol. (i.e. the fact that pd only supports unidirectional messaging creates the awkward concept + of a "passing packet" to eliminate excessive data copying.) + + <p> In pd, the pdp messaging protocol is implemented as pd messages. The protocol is however 3 phase. + With a read only register phase, a read/write register phase and a process phase. This functionality + is part of the base class or the forth processor object. The dpd protocol is entirely different, + and is used in the opengl library. It is + not based on parallel dataflow but serial context passing. + + <h2> Packets </h2> + <p> PDP introduces a new atom: the data packet. This can contain all kinds of data. Images (16bit/8bit), cellular + automata (1bit), matrices (real/complex float/double), opengl textures and 3d rendering contexts. Packets + are stored in a pool to ensure fast reuse, and to enable sharing. The paradigm is centered around a + combination of an object oriented approach and a dataflow approach. + <p>The methods operating on packets + (pdp_packet_*) are mainly for administrative purposes: memory management (construction, registering, copying) + and getting or setting info. + <p>All processing is done in the pd modules. Processors can be defined using + the forth scripting language, but this is still experimental. The forth system can be accessed + from the guile library. + <p> There is a central mechanism for packet type conversion. This is to facilitate the combination of different + media types. Whenever a packet class is constructed (i.e. in an extension library), a number of conversion + routines should be defined to convert the added type to one or some of the main pdp types. + + + + + + + <h2>PDP API Overview</h2> + + The pdp public api contains only a single class: the packet. (The internal api has more classes, that can be used + too if necessary, but i won't document them.) A packet is a class in pdp. The table below lists the supported methods. + The first argument of a call is a packet id. + + <TABLE border = "1"> + <TR><TH colspan = "2">pdp_packet_* + <TR><TD>new <TD>construct a raw packet (depreciated) + <TR><TD>new_* <TD>construct packet of specific type/subtype/... + <TR><TD>mark_unused <TD>release + <TR><TD>mark_passing <TD>conditional release (release on first copy ro/rw) + <TR><TD>copy_ro <TD>readonly (shared) copy + <TR><TD>copy_rw <TD>private copy + <TR><TD>clone_rw <TD>private copy (copies only meta data, not the content) + <TR><TD>header <TD>get the raw header (t_pdp *) + <TR><TD>data <TD>get the raw data (void *) + <TR><TD>pass_if_valid <TD>send a packet to pd outlet, if it is valid, and mark unused + <TR><TD>replace_if_valid <TD>delete packet and replace with new one, if new is valid + <TR><TD>copy_ro_or_drop <TD>copy readonly, or don't copy if dest slot is full + send drop notify + <TR><TD>copy_rw_or_drop <TD>same, but private copy + <TR><TD>get_description <TD>retrieve type info + <TR><TD>convert_ro <TD>same as copy_ro, but with an automatic conversion matching a type template + <TR><TD>convert_rw <TD>same as convert_ro, but producing a private copy + </TABLE> + + + <p>The pool object methods. All the packets are stored in a central packet pool. + + <TABLE border = "1"> + <TR><TH colspan = "2">pdp_pool_* + <TR><TD>collect_garbage <TD>manually free all unused resources in packet pool + </TABLE> + + <p>The process queue object methods. PDP supports a separate processing thread. + + <TABLE border = "1"> + <TR><TH colspan = "2"> pdp_queue_* + <TR><TD>add <TD>add a process method + callback + <TR><TD>finish <TD>wait until a specific task is done + <TR><TD>wait <TD>wait until processing queue is done + </TABLE> + + <p>The control methods. General pdp control messages. + + <TABLE border = "1"> + <TR><TH colspan = "2"> pdp_control_* + <TR><TD>notify_drop <TD>notify that a packet has been dropped + </TABLE> + + <p> The type mediator methods. + <TABLE border = "1"> + <TR><TH colspan = "2"> pdp_type_* + <TR><TD>description_match <TD>check if two type templates match + <TR><TD>register_conversion <TD>register a type conversion program + + +</TABLE> + + + <p>NOTE: it is advised to derive your module from the pdp base class defined in pdp_base.h + instead of communicating directly with the pdp core + + + + <h2>pdp_base class</h2> + If you want to write a pdp extern, you can derive it from the pdp_base class, instead of t_object. + This class abstracts a lot of the hassle of writing ordinary (inplace) packet processors. The base + allows you to register process callbacks. There are 3 kinds of callbacks: preproc, process and postproc. + The preproc method is called inside the pd thread. This can be used to setup some things that can only + be done inside the pd thread. The process method should do most of the work, and is called from the + pdp processing thread if it is enabled, after the preproc method is finished. You can't use most + of pd's calls in this method. The postproc method is called + from the pd thread after the process method is finished, and can be used to send data to pd outlets. Simple + packet processors only need the process method (packet input/output is handled by the pdp_base class). + + <h2>pdp_imageproc_* modules</h2> + Most of the image processing code is organized as planar 16 bit signed processors. + This is crude and oversimplified, but it helps to keep the code size small and fast + at the same time (platform dependent assembly code is reduced to a bare minimum). These + routines can be used to build higher level image processing objects that are more (cache) + efficient than an abstraction using separate pdp modules. If you plan to write your own image + processing routines, you can use the pdp_imageproc_dispatch_ routine to support all 16bit image + types at once (greyscale, subsampled YCrCb, multichannel planar). This requires you write the + image processing routine as a planar (greyscale) processor using the pdp_imageproc_ + interface. (see pdp_imageproc.h) + + <h2>pdp_llconv call</h2> + Low level image conversion routines. (operating on raw data buffers). You probably won't need this, + since the high level type conversion (pdp_packet_convert_ro/rw) covers most of its functionality. + + + + <hr> + <address><a href="mailto:pdp@zzz.kotnet.org">Tom Schouten</a></address> +<!-- Created: Mon Apr 28 15:35:12 CEST 2003 --> +<!-- hhmts start --> +Last modified: Fri Sep 19 04:52:12 CEST 2003 +<!-- hhmts end --> + </body> +</html> diff --git a/doc/misc/layers.txt b/doc/misc/layers.txt new file mode 100644 index 0000000..a02e481 --- /dev/null +++ b/doc/misc/layers.txt @@ -0,0 +1,222 @@ +pdp 0.13 design layers + components +----------------------------------- + +from version 0.13 onwards, pdp is no longer just a pd plugin but a +standalone unix library (libpdp). this documents is an attempt to +describe the design layers. + +A. PD INTERFACE +--------------- + +on the top level, libpdp is interfaced to pd using a glue layer which +consists of + +1. pdp/dpd protocols for pd +2. process queue +3. base classes for pdp/dpd +4. some small utility pd objects +5. pd specific interfaces to part of pdp core +6. pdp_console +7. pd object interface to packet forth (pdp object) + + +1. is the same as previous versions to ensure backwards compatibility in +pd with previous pdp modules and extensions that are written as pd +externs or external libs. this includes parts of pdp that are not yet +migrated to libpdp (some of them are very pd specific and will not be +moved to libpdp), and pidip. if you intend to write new modules, it is +encouraged to use the new forth based api, so your code can be part of +libpdp to use it in other image processing applications. + +2. is considered a pd part. it implements multithreading of pdp inside +pd. multithreading is considered a host interface part, since it usually +requires special code. + +3. the base classes (pd objects) for pdp image processing remain part of +the pd<->pdp layer. the reason is the same as 1. a lot of the original +pd style pdp is written as subclasses of the pdp_base, pdp_image_base, +dpd_base and pdp_3dp_base classes. if you need to write pd specific +code, it is still encouraged to use these apis, since they eliminate a +lot of red tape involving the pdp protocol. a disadvantage is that this +api is badly documented, and the basic api (1.) is a lot simpler to +learn and documented. 3dp is supposed to merge to the new forth api, +along with the image/video processing code. + +4. is small enough to be ignored here + +5. includes interfaces to thread system and type conversion system + +some pd specific stuff using 1. or 3. + +6. the console interface to the pdp core, basicly a console for a +forth like language called packet forth which is pdp's main scripting +language. it's inteded for develloping and testing pdp but it can be +used to write controllers for pd/pdp/... too. this is based on 1. + +7. is the main link between the new libpdp and pd. it is used to +instantiate pdp processors in pd which are written in the packet forth. +i.e. to create a mixer, you instantiate a [pdp mix] object, which would +be the same as the previous [pdp_mix] object. each [pdp] object creates +a forth environment, which is initialized by calling a forth init +method. [pdp mix] would call the forth word init_mix to create the local +environment for the mix object. wrappers will be included for backward +compatibility when the image processing code is moved to libpdp. + + +B. PDP SYSTEM CODE +------------------ + +1. basic building blocks: symbol, list, memory manager +2. packet implementation (packet class and reuse queue) +3. packet type conversion system +4. os interface (X11, net, png, ...) +5. packet forth +6. additional libraries + + +1. pdp makes intensive use of symbols and lists (trees, stacks, queues). +pdp's namespace is built on the symbol implementation. a lot of other +code uses the list + +2. the pdp packet model is very simple. basicly nothing more than +constructors (new, copy, clone), destructors (mark_unused (for reuse +later), delete). there is no real object model for processors. this is a +conscious decision. processor objects are implemented as packet forth +processes with object state stored in process data space. this is enough +to interface the functionality present in packet forth code to any +arbitrary object oriented language or system. + +3. each packet type can register conversion methods to other types. the +type conversion system does the casting. this is not completely finished +yet (no automatic multistage casting yet) but most of it is in place and +usable. no types without casts. + +4. os specific modules for input/output. not much fun here.. + +5. All of pdp is glued together with a scripting language called packet +forth. This is a "high level" forth dialect that can operate on floats, +ints, symbols, lists, trees and packets. It is a "fool proof" forth, +which is polymorphic and relatively robust to user errors (read: it +should not crash or cause memory leaks when experimenting). It is +intended to serve as a packet level glue language, so it is not very +efficient for scalar code. This is usually not a problem, since packet +operations (esp. image processing) are much more expensive than a this +thin layer of glue connecting them. + +All packet operations can be accessed in the forth. If you've ever +worked with HP's RPN calculators, you can use packet forth. The basic +idea is to write objects in packet forth that can be used in pd or in +other image processing applications. For more information on packet +forth, see the code (pdp_forth.h, pdp_forth.c and words.def) + +6. opengl lib, based on dpd (3.) which will be moved to packet forth +words and the cellular automata lib, which will be moved to +vector/slice forth later. + + +C. LOW LEVEL CODE +----------------- + +All the packet processing code is (will be) exported as packet forth +words. This section is about how the code exported by those words is +structured. + +C.1 IMAGE PROESSING: VECTOR FORTH + +Eventually, image operations need to be implemented, and in order +to do this efficiently, both codewize (good modularity) as execution speed +wize, i've opted for another forth. DSP and forth seem to mix well, once +you get the risc pipeline issues out of the way. And, as a less rational +explanation, forth has this magic kind of feel, something like art.. +well, whatever :) + +As opposed to the packet forth, this is a "real" lowlevel forth +optimized for performance. Its reason of being is the solution of 3 +problems: image processing code factoring, quasi optimal processor +pipeline & instruction usage, and locality of reference for maximum +cache performance. Expect crashes when you start experimenting with +this. It's really nothing more than a fancy macro assembler. It has no +safety belts. Chuck Moore doctrine.. + +The distinction between the two forths is at first sight not a good +example of minimizing glue layers. I.e. both systems (packet script +forth and low level slice forth) are forths in essence, requiring +(partial) design duplication. Both implementations are however +significantly different which justified this design duplication. + +Apart from the implementation differences, the purpose of both languages +is not the same. This requires the designs of both languages to be +different in some respect. So, with the rule of "do everything right +once" in mind, this small remark tries to justify the fact that forth != +forth. + +The only place where apparent design correspondence (the language model) +is actually used is in the interface between the packet forth and the +slice forth. + +The base forth is an ordinary minimal 32bit (or machine word +lenght) subroutine threaded forth, with a native code kernel for +i386/mmx, a portable c code kernel and room for more native code kernels +(i.e i386/sse/sse2/3dnow, altivec, dsp, ...) Besides support for native +machine words bit ints and pointers, no floats, since they clash with +mmx, are not needed for the fixed point image type, and can be +implemented using other vector instructions when needed), support for +slices and a separate "vector stack". + +Vectors are the native machine vectors, i.e. 64bit for mmx/3dnow, +128bit for sse/sse2, or anything else. The architecture is supposed to +be open. (I've been thinking to add a block forth, operating on 256bit +blocks to eliminate pipeline issues). Blocks are just blocks of vectors +which are used as a basic loop unrolling data size grain for solving +pipeline operations in slice processing words. + +Slices are just arrays of blocks. In the mmx forth kernel, they can +represent 4 scanlines or a 4 colour component scanline, depending on how +they are fed from packet data. Slices can be anything, but right now, +they're just scanlines. The forth kernel has a very simple and efficient +(branchless) reference count based memory allocator for slices. This +slice allocator is also stack based which ensures locality of reference: +a new allocated slice is the last deallocated slice. + +The reason for this obsession with slices is that a lot of video +effects can be chained on the slice level (scanline or bunch of +scanlines), which improves speed through more locality of reference. In +concreto intermediates are not flushed to slower memory. The same +principles can be used to do audio dsp, but that's for later. + +The mmx forth kernel is further factored down following another +virtual machine paradigm. After doing some profiling, i came to the +conclusion that the only, single paradigm way of writing efficient +vector code on today's machines is multiple accumulators to avoid +pipeline stalls. The nice thing about image processing is that it +parallellizes easily. Hence the slice/block thing. This leads to the +1-operand virtual machine concept for the mmx slice operations. The +basic data size is one 4x4 pixel block (16bit pixels), which is +implemented as asm macros in mmx-sliceops-macro.s and used in +mmx-sliceops-code.s to build slice operations. The slice operations are +built out of macro instructions for this 256bit or 512bit, 2 or 1 +register virtual machine which has practically no pipeline delays +between its instructions. + +Since the base of sliceforth is just another forth, it could be that +(part of) 3dp will be implemented in this lowlevel forth too, if +performance dictates it. It's probably simpler to do it in the lowlevel +forth than the packet forth anyway, in the form of cwords. + +C.2: MATRIX PROCESSING: LIBGSL + +All matrix processing packet forth words are (will be) basicly wrappers +around gsl library calls. Very straightforward. + +C.3: OPENGL STUFF + +The same goes for opengl. The difference here is that it uses the dpd +protocol in pdp, which is more like the Gem way of doing things. The +reason for this is that, although i've tried hard to avoid it, opengl +seems to dictate a drawing context based, instead of an object based way +of working. So processing is context (accumulator) based. Packet forth +will probably get some object oriented, or context oriented feel when +this is implemented. + + + + diff --git a/doc/misc/overview.html b/doc/misc/overview.html new file mode 100644 index 0000000..6eb0e70 --- /dev/null +++ b/doc/misc/overview.html @@ -0,0 +1,102 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Pure Data Packet</title></head> +<body> + +<h1>Pure Data Packet</h1> + + +<h2>Introduction</h2> + +<p>Pure Data Packet (PDP) is an extension library for the computer music +program <a href="http://www.pure-data.org">Pure Data</a> (PD), by <a href = +"http://www-crca.ucsd.edu/~msp/software.html">Miller Puckette</a> and +others. Its goal is to provide a way to use arbitrary data types (data +packets) as messages that can be passed around inside PD, along side the +standard PD numbers and symbol types. In short it puts any data object on +the same level as a float or a symbol. + +<p>PDP runs on Linux and OSX. The OSX version depends on <a +href="http://fink.sourceforge.net/">Fink</a>, which is not in the "point & +click" stage yet, so setting it up will require some efford. There is no +windows version. The reason for this is simple: i don't use windows myself. +Porting would require writing code for input/output and getting the +libraries PDP depends on to work. If anyone is willing to do this, just let +me know. PDP can run without X Window, using SDL. + +<p> Currently, PDP's focus is on images and video, but there is no reason it +should stay like that. There is limited support for matrix processing +included in the main library (like Jitter or Gridflow). There is an +extension library for 1D and 2D binary cellular automata, opengl rendering +(like Gem). Some plans include audio buffers (like Vasp), ascii packets, +text buffers, ... Finally there's a library that enables you to connect a +scheme interpreter (guile) to PD/PDP. For more image processing objects, +have a look at Yves Degoyon's <a +href="http://ydegoyon.free.fr/pidip.html">PiDiP</a> library. + +<h2>Getting Started</h2> + +If you're used to working with PD, the the documentation and example +patches should be enough to get you started. Have a look at the README file +in the distribution to find out how to compile and setup. The file +doc/reference.txt contains a list of objects. If you have installed PDP +properly, you can just press the right mouse button on an object and select +help to get a help patch. If this doesn't work, look in the directory +doc/objects for a collection of help patches. The directory doc/examples +contains some more demos. The directory doc/objects contains two +abstractions that are used to setup the input and output in the help +patches. You might want to cut and connect some wires to use the +input/output setup that works for you. + +<h2>Packets and Types</h2> + +<p> PDP is centered around the concept of packets and operations on +packets. There are several types of packets. The default type for most +objects is <code><b>image/YCrCb/320x240</b></code>. This is a single video +frame, encoded in the internal 16bit YUV format, measuring 320 by 240 +pixels. Another image type is the grayscale image +<code><b>image/grey/320x240</b></code>. Important notes: All image processing objects that +combine two or more packets need to be fed with the same packet types, i.e. +encoding (YCrCb/grey) and dimensions need to be the same. Image dimensions need to be a +multiple of <code><b>8x8</b></code>. + +<p> The +<code><b>bitmap/*/*</b></code> type is another image representation type +supporting several encodings. I.e. <code><b>bitmap/rgb/*</b></code>, +<code><b>bitmap/rgba/*</b></code>, <code><b>bitmap/yv12/*</b></code>, ... + +This type cannot be processed directly by most of the image processing +objects, but it can be used to store in delay lines, or to send over the +network. It's main use is to support all kinds of input/output devices, and +opengl textures, without introducing too many conversions, but it can serve +as a space and bandwidth saver too (especially +<code><b>bitmap/yv12/*</b></code>). + +<p> One of the interesting +features in PD is the possibility of connecting everything with everything. +If you want to generalize this to all kinds of media objects, the complexity +of managing the different types starts to grow quite fast. Therefore PDP has +a type conversion system that can take care of most of the conversions +using the <code><b>[pdp_convert]</b></code> object. You can manually convert +packets to a certain type by specifying a type template as a creation +argument. I.e. <code><b>[pdp_convert image/grey/*]</b></code> will convert +any packet to a greyscale image. Most of the conversion will become +automatic later on. + +<p> An example: You can use the basic PDP library together with the +cellular automata library and the opengl rendering library to use a cellular +automaton as an input to a video processing chain. You can convert the +processed image to a texture that can be applied to a 3d object, which then +can be drawn to the screen, captured as a texture, converted back to an +image, which can then be converted to a sound, processed and converted back +to an image, etc... You get the point. The possibilities are endless. + + + + <hr> + <address><a href="mailto:pdp@zzz.kotnet.org">Tom Schouten</a></address> +<!-- Created: Thu Apr 24 22:21:03 CEST 2003 --> +<!-- hhmts start --> +Last modified: Thu Sep 25 20:51:44 CEST 2003 +<!-- hhmts end --> + </body> +</html> diff --git a/doc/misc/todo.jme b/doc/misc/todo.jme new file mode 100644 index 0000000..2ce317d --- /dev/null +++ b/doc/misc/todo.jme @@ -0,0 +1,4 @@ +todo list of jme@off.net +------------------------ +- a packet to trigger packet generator instead of bang + o the created packet has the same format as the incoming packet |