diff options
Diffstat (limited to 'opengl')
| -rw-r--r-- | opengl/Makefile | 27 | ||||
| -rw-r--r-- | opengl/Makefile.config | 25 | ||||
| -rw-r--r-- | opengl/README | 239 | ||||
| -rw-r--r-- | opengl/TODO | 121 |
4 files changed, 412 insertions, 0 deletions
diff --git a/opengl/Makefile b/opengl/Makefile new file mode 100644 index 0000000..d62358e --- /dev/null +++ b/opengl/Makefile @@ -0,0 +1,27 @@ +include Makefile.config + +all: $(TARGET) + +linux: pdp_opengl.pd_linux + +darwin: pdp_opengl.pd_darwin + +subdirs: + make -C system + make -C modules + make -C include + +clean: + make -C system clean + make -C modules clean + make -C include clean + rm -f pdp_opengl.pd_linux + rm -f *~ + +pdp_opengl.pd_linux: subdirs + rm -f pdp_opengl.pd_linux + $(CC) -export_dynamic -shared -o pdp_opengl.pd_linux modules/*.o system/*.o $(LDFLAGS) -g + +pdp_opengl.pd_darwin: subdirs + rm -f pdp_opengl.pd_linux + $(CC) -o pdp_opengl.pd_pd_darwin modules/*.o system/*.o $(LDFLAGS) -g -bundle -bundle_loader $(PD_EXECUTABLE) diff --git a/opengl/Makefile.config b/opengl/Makefile.config new file mode 100644 index 0000000..4e59a4c --- /dev/null +++ b/opengl/Makefile.config @@ -0,0 +1,25 @@ +PD_DIR = /home/tom/pd/distro/pd/src +PDP_DIR = /home/tom/pd/packet/include +PDP_OGL_DIR = /home/tom/pd/packet/opengl/include + + + +CFLAGS = -DPD -O2 -funroll-loops -fomit-frame-pointer -ffast-math \ + -Wall -W -Wstrict-prototypes -Werror \ + -Wno-unused -Wno-parentheses -Wno-switch -g + +CPPFLAGS = -I$(PD_DIR) -I$(PDP_DIR) -I$(PDP_OGL_DIR) -I/usr/X11R6/include +LDFLAGS = -lGL -lglut + +TARGET=linux + +#uncomment these for darwin: +#TARGET=darwin +#CPPFLAGS+=-I/sw/include +#PD_EXECUTABLE=/usr/local/bin/pd +#LDFLAGS = -lGL -lGLU -lglut -lX11 -L/sw/lib -L/usr/X11R6/lib + + + +.c.o: + $(CC) $(CFLAGS) $(CPPFLAGS) -o $*.o -c $*.c diff --git a/opengl/README b/opengl/README new file mode 100644 index 0000000..fb756b3 --- /dev/null +++ b/opengl/README @@ -0,0 +1,239 @@ +pdp_opengl (3dp): opengl extensions for pdp +warning: this is still experimental and incomplete + +this library extends pdp with texture and render context packets, +to use some of the power of current video hardware. + +it is very much like gem (sort of a redesign, filled with my own +idiosyncrasies). gem, like it is now, is not very suited for interaction +with pdp image processing because of its hardcoded window centric rendering, +so i thought i'd write some gem like stuff on top of pdp myself. (3dp right +now only supports window centric rendering itself, but adding pbuffer +or pixmap rendering is easy enough, though it requires a rather new glx +library) + +so, 3dp is an experimental gem clone with tight pdp integration. unless +you like experiments, use gem, since it is better supported, has more +features and runs on linux, windows and mac osx. + +requires glx (opengl on x window). + +building: + +edit Makefile.config to reflect your system and run make. the library is +pdp_opengl.pd_linux + +some of the examples use the abstractions in opengl/abstractions. best to +add this to your pd path. + + +there are some fatal X bugs floating around, so be careful with resizing +windows and closing patches. there are a lot of other, non-fatal bugs +or badly implemented features. if you have remarks, just let me know. + + +i'll move to autoconf once i no longer consider it experimental. + + +TIPS & TRICKS + + +* the 3dp_windowcontext creates a window to draw in. the window packet +will be be output to the left outlet when a bang is received. control +flow for context is right to left, this means if a 3dp object has 2 +context outlets, the rightmost will be propagated before the leftmost. +there is no fanout. all operations are accumulative, including the +geometric transformations. if you need to branch use a 3dp_push object. + + +* geometric transformations can be done using the 3dp_view object. +the first argument is the kind of transformation: scale, scalex, +scaley, scalez, rotx, roty, rotz, rota, transx, transy, transz, +transxyz. + +repectively this scales the object, the x, y, z axis, rotates +around the x, y, z axis, rotates around an arbitrary axis, translates +in the x, y, z directions and translates along a vector. the initial +parameters can be specified as creation arguments, i.e.: + +[3dp_view rota 1 1 1 90] rotates by 90 degrees around axis (1,1,1). + + +* some simple objects can be drawn using the pdp_draw object. the +first argument is the object: square, cube, sphere, torus, cone, +teapot, dodeca, icosa, octa, tetra. prepending the names with a w +draws the wireframe version. (i.e. wcube is a wireframe cube). the +first inlet of the object is a texture inlet. not all objects support +this, but cube, square and sphere do. other inlets set some parameters +like size, nb of segments for sphere, etc.. they can be specified +by creation arguments too. + + +* saving a (matrix) state can be accomplished by the 3dp_push object. +the default matrix is the modelview matrix. it works as follows: both +the right and the left outlet propagate the same matrix state as the +input. so in short you can use 3dp_push to split your rendering tree +into parallel branches. the matrix types that can be pushed are: +modelview, texture, color, projection. + +* setting a current matrix can be done using the 3dp_mode object. +i.e. [3dp_mode texture] will map all geometric transforms to the +texture matrix, for texture coordinate animation. the left outlet +restores the current matrix back to the modelview matrix. + +* it is possible to send a render context trough a drawing/view +transforming object multiple times. the easy way is to use 3dp_for, +but other ways are legal too. this can be done to draw different +versions of the same object. have a look at example01 how this can +be done. + +it is also possible to send multiple render contexts trought the same +rendering tree (i.e. multiple windows). if you use different viewing +transformations on the top of the rendering chain, you can view a scene +trough different angles. + + +* light sources can be introduces using 3dp_light. they can be moved +with oridinary 3dp_view objects. + + +* 3dp_color changes the current color. right outlet is new color, +left outlet is the previous color. + +* 3dp_toggle toggles a boolean opengl switch. enabled now are: +depth_test, blend_add, blend_mix. more to come later. + + + +* couping 3dp and pdp can be done using 3dp_snap and pdp_convert. +the correct way to do it is to put 3dp_snap in a rendering +chain and give it arguments like this: + +[3dp_snap image/*/* 320 240] + +if you specify the subtype to be image/multi/*, the packet +will not be colour space converted: it will stay rgb. +if you want to make a snapshot to store as a high quality +png image, snap to bitmap/rgb/* and store it in pdp_reg to save. +to convert an image back to a texture, use + +[pdp_convert texture/*/*] + +if you snap to a texture (which is the default) +the dimensions don't need to be specified. a texture will be +allocated that can contain the entire screen. this is because +texture coordinates are relative and data is always interpolated. + +snapping only works correctly when the window is not covered +by other windows. + + +* textures can have any dimensions, but only those which have +dimensions that are integral powers of two will be tiled correctly. +i.e. by using pdp_mode to change to texture mode and doing some +coordinate transforms using pdp_view (see example06: this +uses a tilable animated cellular automata texture) + + +* multipass rendering is supported trough the objects +3dp_subcontext, "3dp_draw clear" and "3dp_view reset". the idea +is as follows: before rendering the final scene, you use +(a part of) the drawing buffer to render some things and store +them in a texture to be used in your final drawing pass. have +a look at examples 11, 12 and 13. in theory you could build a +"texture processing" framework on top of 3dp, by using the window +buffer as an accumulator and using textures as auxilary registers, +and drawing your final texture result using i.e. +3dp_display_texture. while this can be much faster than ordinary +pdp image processing, it also requires you to do more bookkeping +yourself, since you can only use a "serial" approach because +you can't modify textures directly, only trough the use of the +render buffer. + + +* 3dp has it's own thread, which is enabled by default. you +can enable/disable this by sending a "3dthread 0/1" message +to pdp_control. in most cases there's no reason to disable +the thread processing, except when you are doing a lot of +pdp<->3dp conversions. in that case the delay introduced by +the thread processing might become problematic. (the same +goes for pdp btw. feedback and threads don't mix too well). +it's a tradeoff. thread processing gives priority to the audio, +so you can obtain low latency, and you don't have to care +about locking up pd by doing too much processing. (instead +frames will be dropped). the drawback is of course that video +timing becomes unpredictable because it is governed by the system +scheduler. so sometimes it can be useful to disable threads +and increase the audio latency, so you can have snappy audio +and video at the same time. getting this to work well usually +requires some experimenting. + + +enjoy, + +tom + + + + +--- +some political ranting: gem vs. pdp/3dp + +first a disclaimer. i'm not into dissing people. i respect and appreciate +all the work that has gone into gem, but at the same time i'm not too happy +with what gem has become. not so much the functionality, but the way it is +built. the original design as a 3d processing extension is not flexible enough +to incorporate what's in there now and what could be added in the future.. + +instead of complaining about it, i decided to be pragmatic and write something +from scratch. i think, sometimes this has to be done. for me writing pdp/3dp +has been an extremely interesting learning experience. i think i understand +the trade-offs better now, and maybe these remarks can be useful. + +opengl is not a pure dataflow language. it operates on a global machine +state next to the obvious drawing buffer that is passed around. +representing opengl code with a graphic tree view has some advantages, +though it has a different "feel" than normal pd patches. the fact that +opengl transforms object coordinates to screen coordinates, and not vice +versa, gives graphicly represented opengl code an "upside down" feel. + +one of the things i don't like about gem is that it has both the opengl +"upside down drawing and coordinate transformation tree" and the pix +"data flow image processing tree" in the same network, which i find +counterintuitive. it is too monolytic to be truly flexible. in pdp/3dp +i try to separate the two explicitly: dataflow where possible (image +processing), and serial context based processing where needed (opengl). + +another disadvantage of gem is its window centric context. by building +3dp around explicit context passing, with explicit context creation objects, +i try to avoid this. an advantage of this is the possibility to send +multiple contexts trough a rendering tree, i.e. to have 2 different views +of the same scene. + +pdp has implicit fanout for all packets. i think that is its great +strength. it enables you to use any kind of media packet like you would +use floats. it is very intuitive. however, 3d rendering does not fit +this shoe. at least not when you just wrap opengl in the most straightforward +way. 3dp is a test case for pdp's "accumulation packets", which is a formal +abstraction of a context. pdp processors are noting more than serial programs +working on a context, so in fact it's nothing special. in fact, i'm still +in doubt if it is useful besides being able to support opengl.. + +so, the idea was to improve upon gem a bit, from different angles. i did +not succeed in my primary goal: making 3dp more intuitive than gem. it seems +this is only possible if you limit some of the flexibility. just wrapping +opengl keeps a lot of flexibility, but get's infected with the opengl +paradigm. by separating pdp (image processing) and 3dp (drawing and geometry +processing) as much as possible i think i've solved some intuition problems, +but 3dp itself is still basicly gem, with imho a better overall design, +but due to its more low level approach, maybe harder to understand. it seems +to me that knowing how opengl works is rather necessary to use 3dp. the same +is true for gem. + +one last philo remark: to me it seems the biggest drawback of gem's design is +the processor centric aproach: the only objects are processors, the data is +implicit and seemingly global. in pdp/3dp processors and objects are 2 separate +entities. i tried to unify them in one object model, but to me it seems the two +should really be treated as different complementary entities, to stay close to +the dataflow paradigm which makes pd such an intuitive tool. + diff --git a/opengl/TODO b/opengl/TODO new file mode 100644 index 0000000..dc61332 --- /dev/null +++ b/opengl/TODO @@ -0,0 +1,121 @@ +bugs: WARNING: there are still fatal bugs lurking around. + +* segfault when combining 3dp_windowcontext and pdp_xv. probably a mistake +in the teture<->image conversion or window event handling. +* 3dp is not robust against running out of video card resources. if you +manage to crash it please consider sending a bug report. +* 3dp_dlist triggered a segfault in pdp_mutex(un?)lock. can't reproduce. +this also happens on some other occasions where a post() statement is involved: +crash in pdp_mutex_lock, called by putc(). +* pd hangs on save? what happens during save?? -> 0.35 prob? 0.36 seems to work fine. + + +todo: + +* fix flow control for texture conversion +* fix display event routing for multiple windows +* finish mesh object + + +general: + +* prevent display list recursion: add some state data to the context packet to solve this. + +redesign: +* unify pbuf & window contexts (postponed pbufs because of subcontexts) +* cube mapping +* bubble object (model + wave equation) +* finish 3dp light + +performance: +* why is texture upload so slow? and is there a way to work around it? +* (why) is context switching slow? and how to introduce the render context differently. (direct to window?) + + + +DESIGN NOTES + +3dp is basicly my idea of gem on top of pdp. it is a simple layer around opengl. +3dp_* render and transform objects accept a reference to a rendering context and pass this along. + +3dp_* objects DO NOT SUPPORT FANOUT. they do support fanin. multiple contexts can be sent to an object, which +will result in drawing in (or other manipulations of) the respective contexts. + +texture packets can be used to pass bitmap data around. 3dp_tdraw accepts a texture on its +second inlet. 3dp_snap dumps the current state of the buffer into a texture packet. + +object classes: + +-drawing objects + +[3dp_draw cube] [3dp_draw sphere] .. + +-manipulation objects + +[3dp_view rot2d] + +-opengl stack objects + +[3dp_push color view texture] + +3dp vs pdp design: + +a context packet is an accumulation packet, and the order of operations on it (serial) is important. it has +elements from a readonly packet (only one instance is passed around) and from a rw packet (it is legal to change it's +state and pass it on). + +opengl in dataflow formulation seems to be read bottom to top. i first thought +this was a big drawback of gem, but now that i finally understand how opengl works i think it is +ok. it suddenly dawned on me, usually the number of eyes is much smaller than the number of objects +in a scene, so it is much more straghtforward to start at the eye instead of the objects. since a +simple serial stack mechanism can be used. + +so opengl is really serial, while pd "looks" parallel, but is serial too. i still think this +is a good thing, since it keeps things simple, but for some reason the "upside down & serial" +thing about opengl in pd is rather strange.. + +once you get used to it, and think about rendering a set as a tree rooted at the "context source" +like is done in gem,it seems to work out.. + +i think i'm going to stick to the depth first backtracking a tree serial rendering +way of looking at it. since i don't see a way of making this more intuitive without complicating +it too much. so no legal fanout of a context packet. + +------------------ + +accumulation packets (buckets) & backtracking + +buckets add support for "context based" single threaded programs. it +basicly allows you to construct data processors represented as a control +flow tree. i.e. function calls can be represented by branches in a tree, +with each branch rooted at an object's outlet, executed from right to +left. a "context packet" (or accumulation packet or bucket) is passed +along and acted upon. + +* fanout is illegal for bucket processors (at least, if they are non +cummutative, as is usually the case). this is not explicitly prohibited, +since it is hard to check in pd and it allows some hacks. the reason for +this is obvious: if you do not know which branch of a tree is executed +first, results are undefined. + +* a new communication protocol needs to be introduced, incompatible +with the existing 3 phase protocol: + +(1) register_ro +(2) register_rw +(3) process + +the new dpd (bucket / accumulation packet) protocol is 2 phase: +(1) inspect +(2) accumulate + +(1) is only present to give priority to bucket inspectors over +accumulators (processors) + +an accumulation packet will be owned by its creator all the time. +accumulation processors do not increase the reference count. only ro +inspectors will. + +note: it is legal for a packet to have a broken register_rw (missing +copy constructor, i.e. if a copy is too expensive or impossible. this +must be set in the pdp packet header flags by the packet constructor) |