diff options
Diffstat (limited to 'externals/gridflow/TODO')
| -rw-r--r-- | externals/gridflow/TODO | 431 |
1 files changed, 431 insertions, 0 deletions
diff --git a/externals/gridflow/TODO b/externals/gridflow/TODO new file mode 100644 index 00000000..c6a1818a --- /dev/null +++ b/externals/gridflow/TODO @@ -0,0 +1,431 @@ +/* $Id: TODO 3975 2008-07-04 19:57:32Z matju $ */ + +0.9.5 +[ ] get rid of architecture.html +[ ] support "get" in abstractions +[ ] fix [nbx]<->[hsl] relationship in [#camera] +[ ] bugs roman +[ ] add claude's fixes +[ ] other gf mails in inbox +[ ] #line_to_polygon is buggy +[ ] #draw_polygon init_lines() not sufficiently called +[ ] regular dc1394 options +[ ] irregular dc1394 options: white_shading, white_balance + trigger_mode? trigger, trigger_delay +[ ] Accumulation of Background Statistics +[x] correction/détection de translations dans l'image (avec fft et #remap_image). +[ ] cvEstimateRigidTransform +[ ] compenser pour les ajustements progressifs ou soudains de luminosite +[x] ajouter detection et correction d'interference par fft +[ ] cvEllipse +[ ] cvApproxPoly pour un seul poly `a la fois +[ ] cvMeanShift (?) +[ ] cvCamShift (?) + Ça s'utilise presque comme MeanShift, sauf que ça requiert un autre type non-grid. + Celui-ci peut être représenté comme une grid de 5 éléments pour faire plus facile + (contrairement à cvHistogram, qui contient une grid ND mais a besoin de quelques fields de plus que ça). +[ ] cvCalcBackProject (?) +[ ] cvCalcHist, cvCreateHist. +[ ] on doit aussi commencer à rajouter des types non-grid. Le premier sera CvHistogram et est requis pour tous les objets dont le nom contient "Hist". +[ ] SnakeImage (what is the concept of 'energy', squared luminosity?) + J'aurais supposé que oui, mais c'est juste trois mesures reliées à trois forces qui "shapent" la courbe: + espacement égal des points, réduction des angles, et correspondance avec l'image même (qui doit normalement + être un edge-detect d'une image plus vieille). +[ ] cvOpticalFlow + +whatever version +[ ] [cv.split] +[ ] reestablish 'gzfile' feature in a better way. (gzfile is disabled for now) +[ ] ... +[ ] [fps]: translate into pd: "publish if fps>0.001" (to remove outliers) +[ ] conversion from float to int is inconsistent: when to round and when to trunc? +[ ] reimplement negative indices in [listelement] and [listsublist] +[ ] with pwc, 720x480 crashes +[ ] fix SDL modifiers +[ ] fix quoting in [display] + +soon: + * remove reference.*ml + * videodev.c should be upgraded to V4L2 asap. + ++-------+-------+-------+-------+-------+-------+-------+------- +Common Video Framework : + +[ ] a common library for all video IO (cameras, files, web), which is the + project of Thomas H and Johannes. + +[ ] a six-way converter for GEM/PDP/GridFlow. There are already + converters between GEM and PDP and between GEM and GridFlow but + there is none between PDP and GridFlow and then some things could + be easier to install and run in a cross-platform way. + +[ ] a way to make GEM/PDP/GF conversions *implicitly* instead of + having objects for that. + +[ ] threads for video? +[ ] GEM geos vs 3DP: how do they compare? +[ ] is there anything else deserving to be unified between the video plugins? +[ ] videojack +[ ] http://gstreamer.freedesktop.org/ +[ ] distinguish between C++ types Grid* and GridFlow* +[ ] make [#draw_points] [#draw_rect] [#edit_polygon] +[ ] Salsaman recommends supporting Videojack because of unicap support which means getting DV support for free. +[ ] look at camorama +[ ] OpenCV +[ ] Generic Image Library: http://opensource.adobe.com/gil/html/giltutorial.html +[ ] [for] has a bug in "from". +[ ] #out x11 doesn't accept 0-sized grids +[ ] #print doesn't accept 0-sized grids +[ ] #inner doesnt support seeds of size>1, and isn't vecop-compatible, and ... +[ ] #inner could optimise permutation matrices? +ClaudiusMaximus matju: [#draw_image] inlet 0 method "op <opname>" is undocumented, but very useful :) +ClaudiusMaximus matju: [unix_time] is maldocumented, outlet 0 is a grid Dim[28](uint8), not a symbol + +[ ] update doc +[ ] claude wants http://en.wikipedia.org/wiki/Graham_scan + +malloc alignment = 4 mod 8 (en utilisant valgrind) + +ClaudiusMaximus matju: feature request: [#store4], behaves like store but accepts grids +of floats as coordinates and interpolate using 4-point interpolation, eg, right inlet -> +(3 3 f # 0 0 0 1 1 1 2 4 8), left inlet -> (1 # 0.5) outputs (0.5 0.5 0.5) +(or whatever the interpolation would really be), left inlet (2 # 1.5 1.5) outputs (3) + +http://oprofile.sourceforge.net/about/ +also: qprofile, amd profiler, opreport -cl --demangle=smart + +[ ] -ftree-vectorizer-verbose +[ ] try removing or modifying -falign on PPC/OSX +[ ] encoding of quicktime RGBA only works with [#cast b] (is that still true?) +[ ] reading tux.tga is completely broken +[ ] install of libquicktime is very wrong on my system (is it 0.9.8?) +[ ] make example on separable convkernels +[ ] add concepts of hot inlet and conf inlet at the code level +[ ] [display] should use clocks to update tk less often +[ ] [display] leaks in server ? +[ ] deprecation warnings +[ ] Add a video latency patch (in examples) +[ ] Numop: warn on misalignment +[ ] remove GridInlet* ? +[ ] configure chokes if GCC3 doesn't have the C++ plugin +[ ] [#import~] +[ ] [#export~] +[ ] [#expr] +[ ] [#line]? [#pipe]? [#delay]? +[ ] #: option swap +[ ] consider including [mysql] [renamefile] [ls] [shell] +[ ] re-enable int64,float64 +[ ] added #matrix_solve, #slice +[ ] Copy-on-Write (COW) +[ ] internals documentation +[ ] internal grid protocol redesigned +[ ] raise limit of in/outlets from 10 to 100 or unlimited. +[ ] look into -ftree-vectorize +[ ] look at Tetra Piccolo capture cards, as well as ImpactVCB (Hauppauge) + +[145] "list -1"->[@import] only tells error message once every other time +[144] small leak remaining in x11. may have to do with non-use of XDestroyImage. +[140] is [@convolve] actually cross-convolution? + that is, convolves with a mirror image of the convolution matrix. +[139] [peephole] + <mis> matju: if you try to move the GOPed abstraction the peepholes don't move along + <mis> he :) better yet, the abstraction's bounding box's lower right corner will not + move up or to the left of the low right corner of the peephole it contains. + Instead it will resize itself as you move it around. +[134] #import with zero size crashes? +[133] numop ** can freeze GF +[117] osx: [#store] crash instead of raising +[107] MMX computations may be done misaligned, which may be bad. +[105] when file not found, mpeg error messages are wrong, they say the filename is "". +[102] osx: error: shmget failed (cannot allocate memory) in X11, for regular pictures +[078] "unsupported codec" bogus errors. +[025] format_grid: write test for other bpv's +[030] FormatGrid: not properly tested (crashes sometimes) +[031] stdout should be redirected if bridge is present +[034] format/mpeg3.c,quicktime.c might cause lockup on exit +[038] some seeds aren't typechecked +[039] jpeg file cat crashes on second file (jpeglib buffering?) +[040] GRID_FINISH triggered while backstore ??? +[079] can trap segfaults in codecs of libquicktime. +[090] http://poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC30 +[041] what would be a way to make GF work well with Pd's idea of a list? +[045] write unit tests about required functionality +[046] write unit tests about error checking +[047] make system to ensure everything is working before a release. +[048] potential issues between fork() and X11 ? +[054] it's possible to crash by sending data in two inlets of a same object at once + +--- make the development process more test-driven: + * file every feature request and bug report into base/test.rb so + that gridflow fails those tests THEN make gridflow pass those tests. + +=================== + +Installation +[ ] less compile-time checks and more run-time checks +[ ] should GridFlow be bundled with some of its own dependencies? +[ ] find a good profiler (not gridflow's internal...) +[ ] cross-compile: + http://www.libsdl.org/extras/win32/cross/ + http://rooster.stanford.edu/~ben/linux/cross.php + +Documentation +[ ] make a short feature list as a promo +[ ] add credits +[ ] document how to report bugs, basic troubleshooting +[ ] document our distribution of the archives +[ ] ascii conventions: [foo] vs "foo", etc. +[ ] document internals +[ ] describe indexmap in doc +[ ] document backward compatibility issues +[ ] architecture: class->type/shape->use->role +[ ] heavily hyperlink documentation +[ ] automate more parts of the documentation +[ ] benchmarks (especially: simple video i/o) +[ ] mimic [pdp_help_input] [pdp_help_output]: they are convenient +--- remember to keep doc/help/screenshots/examples up to date + +Architecture +[0.9] reentrancy of objects (to allow true recursion) +[0.9] inlet-attrs (attrs associated to inlets) +[ ] eliminate the Dim class, replacing by Grid itself (?). +[ ] pull mode (functional-style) +[ ] bidirectional messages (constraint/relational-style) +[ ] send-reply messages (imperative-style) +[ ] add reflection (#global:classes, #global:methods, ...) +[ ] transmitting grids by diffs? +[ ] dispatch on _#{i}_-named methods (any selector one inlet) +[0.9] generalize the backstore +[ ] Outlet#buffer should harmonise with the lcm of its corresponding Inlet#factor's. +[ ] lazy grids: Grid#operator T* makes sure all the grid is received + and Grid#range(int start, int n, T bogus) makes sure the + specified part of the grid is received. +[0.9] accounting: grids messages packets numbers bytes copybytes +[0.9] memory profiling +[ ] add a "query" class for debugging other objects? + (would be able to extract extra info from each object/class) +[ ] add a GridFlow::LaterException or something + (which tells to retry later with the same packet because another + grid reception is not completed yet.) +[ ] interpret nested lists as n-dimensional. +[0.9] MMX memcpy() ? +[0.9] MMX _runtime_ autodetection + +Computation Objects +[ ] #counter (similar to #for) +[ ] #for "step" argument should have default {1...} +[ ] zero option in #convolve (?) +[ ] #conv (simpler #convolve) should allow n-D convolution +[ ] #convolve rewritten as combination of #conv+#border ? +[ ] #store: option zero (?) +[ ] #import~, #export~ +[ ] #fold, #scan: option reverse (right-to-left) +[ ] #join: option swap +[ ] #matrix_inverse, #matrix_solve +[ ] #timeline (or rtmetro): sequencing of events from grid +[ ] #replace_if, #random_select, #markov_chain, #sort, #slice, #splice +[ ] #polygon_perimeter, #polygon_area, #draw_polygon smoothly +[ ] fuzzy #store (interpolate) +[ ] write a complete benchmark script +[ ] spatial biquads (ask Tom) +[ ] string operations to compensate for the feebleness of PD in that area. + this includes [#sprintf]. +--- see also APL/J/NArray/PDL/Aplus/Yorick/IPOW reference + +Effects +[ ] #mosaic +[ ] #gaussian_blur, #sobel, #pixelize, #despeckle (median) +[ ] #dither, #draw_lines, #draw_points, #fire, #queue +[ ] #chroma_key: i0=RGBpicture i1=RGBcolor i2=tolerance o0=RGBApicture + with fuzzy flag; (don't call it #chroma_key though) + maybe #mask... +[ ] turtle (logo style) +[ ] effectv +[ ] create abstractions for usage patterns (eg: like #remap_image) +[ ] see gimp for ideas +[ ] kaleidoscopes +[ ] bilinear interpolation a+(b-a)*x+(c-a)*y+(a-b-c+d)*x*y (#scale_by smoothly?) +[ ] #scale_by/#downscale_by fractional values +[ ] bicubic interpolation +[ ] multilinear interpolation (n-dimensional?) +[ ] #daltonism +[ ] #perlin ? +[ ] #moment could support more than 2 dimensions + +#<vektor> told me to: +# RGBtoYUV : @fobjects = ["#inner (3 3 # 66 -38 112 128 -74 -94 25 112 -18)","@ >> 8","@ + (16 128 128)"] +# YUVtoRGB : @fobjects = ["@ - (16 128 128)",#inner (3 3 # 298 298 298 0 -100 516 409 -208 0)","@ >> 8"] + +Input/Output +[ ] a #store that is persistent (saved in .pd) ? +[ ] Grid: support storage in all six types +[ ] Grid: bpv 1,2,4 uint +[ ] Grid: text version of the format +[ ] Gimp, GStreamer, ffmpeg, xine, mplayer, PDP, OSC +[ ] get subtitling of tv channels +[ ] add messages "load", "save" as open,bang,close combos. (?) +[ ] x11,sdl,aalib: use buffering to synchronise frames at a constant rate. +[ ] x11: sync with the actual vsync of the monitor +[ ] x11: double buffering using a Pixmap remote-object +[ ] [serial_port] +[ ] support for audio in the mpeg streams +[ ] synchronized audio with the video when using mpeg1 & 2 +[ ] full screen mode (without the menu bars or frames) +[ ] support for audio in cameras / videodev ... see field vcaps.audios + +Format +[ ] remaining number type support in formats +[ ] fork: pseudoformat for launching format handler as separate process +[ ] put_at/reassign in x11/sdl/quartz +[ ] Generic SHM/MMAP Drivers +[ ] "open grid pipe" and such. +[ ] multi: multi-file (by glob pattern, e.g.: foo*.ppm) +[ ] AVI +[ ] format grid: "headerless until 10" (text files) + +User Interface +[ ] color picker +[ ] #edit, grid edition dialog, etc. +[ ] vector slider (n parallel sliders in one, Dim(n)) +[ ] 2d slider +[ ] radial slider (angle chooser) +[ ] drag-and-drop palette of object construction +[ ] object construction menu specialized for each class (generated from class reflection/metadata) + +Extra/Examples/Etc +[ ] .pd: read; write better +[ ] examples/generique.pd +[ ] add examples from the workshop, etc + +<matju> alx: +le tab completion ne donnerait pas seulement les noms possibles pour le +prochain argument dans le objectbox; il donnerait aussi une description de +l'argument (surtout dans le cas où l'argument a beaucoup de valeurs +possibles). je pense que le help de Pd est une bonne idée (le concept hands-on) +mais ça pourrait aller plus loin. j'ai pensé intégrer le manuel de référence de +gridflow directement dans le programme, disponible au runtime, dans l'éventualité +d'un GUI qui saurait en tirer avantage. + +<alx1> matju: détecter les formes; détecter les couleurs; support mpeg4; hotspots + +ClaudiusMaximus reports that allocating a 197M grid causes a crash (turns out C++ +throws exception, can't allocate, signal ABRT) + +ClaudiusMaximus says: I want to be able to send output from Pd+Gridflow to a video4linux +loopback device so that other v4l programs can access the stream. +I found the video loopback device here: http://motion.sourceforge.net/vloopback/ + +<alx2> +iSight on linux : http://episteme.arstechnica.com/6/ubb.x?a=tpc&s=50009562&f=8300945231&m=40110681 +needs something called 'coriander'. the person who got it working mentions libdc1394. +http://www.tele.ucl.ac.be/PEOPLE/DOUXCHAMPS/ieee1394/cameras/index.html: +I learn that the iSight is compliant with version 1.30 of IIDC. +the docs for coriander mention that the author needed to create a major and minor device for the camera. +retyping (damn pdf's) -> there are two devices to create: raw1394 and video1394 +raw1394 has major 171 and minor 0 while video1394 had major 171 and minor 16 +any useful info? +having a camera like the iSight supported could be fantastic for video capture/detection on a laptop. Not to mention that the bttv cards are getting harder to locate + +-mfpmath=sse2 + +remember to check for volatile VALUEs if GC problems ever occur + +Tom Schouten wrote: " +added demo/rawvideo.pf and some support scripts in bin/pf-rawvideo.* +this is an example on how to use external programs for raw video input. +rationale: +mplayer is a very nice piece of software supporting virtually any input +format/codec. since i'm not planning to waste time on integrating more +input codecs in pf, the best alternative is to have mplayer (actually +mencoder) or ffmpeg as a subprocess. +the same will be done for recording later. +" + +very old bookmarks: + http://pdl.perl.org/ + http://www.opendragon.com/Pages/MaxObjects.shtml (not responding) + gstreamer + Arts Builder + http://homepages.inf.ed.ac.uk/rbf/HIPR2/wksheets.htm + http://freej.dyne.org/ + +FObject.subclass("rubyarray",2,2) { + def initialize() @a=[]; @i=0; end + def _0_get(s=nil) + case s + when :size; send_out 1,:size,@a.size + when nil; _0_get :size + end + end + def _0_clear; @a.clear end + def _0_float i; @i=i; send_out 0, *@a[@i] if @a[@i]!=nil; end + def _1_list(*l) @a[@i]=l; end + def _0_save(filename,format=nil) + f=File.open(filename.to_s,"w") + if format then @a.each {|x| f.puts(format.to_s % x) } + else @a.each {|x| f.puts(x.join(",")) } + end + f.close + end + def _0_load(filename) + f=File.open(filename.to_s,"r") + @a.clear + f.each {|x| @a.push x.split(",").map {|y| Float(y) rescue y.intern }} + f.close + end +} + + + +FObject.subclass("regsub",3,1) { + def initialize(from,to) _1_symbol(from); _2_symbol(to) end + def _0_symbol(s) send_out 0, :symbol, s.to_s.gsub(@from, @to).intern end + def _1_symbol(from) @from = Regexp.new(from.to_s.gsub(/`/,"\\")) end + def _2_symbol(to) @to = to.to_s.gsub(/`/,"\\") end + #doc:_0_symbol,"a string to transform" + #doc:_1_symbol,"a regexp pattern to be found inside of the string" + #doc:_2_symbol,"a replacement for the found pattern" + #doc_out:_0_symbol,"the transformed string" +} + +FObject.subclass("memstat",1,1) { + def _0_bang + f = File.open("/proc/#{$$}/stat") + send_out 0, Float(f.gets.split(" ")[22]) / 1024.0 + f.close + end + #doc:_0_bang,"lookup process stats for the currently running pd and figure out how much RAM it uses." + #doc_out:_0_float,"virtual size of RAM in kilobytes (includes swapped out and shared memory)" +} + +FObject.subclass("rubysprintf",2,1) { + def initialize(*format) _1_list(format) end + def _0_list(*a) send_out 0, :symbol, (sprintf @format, *a).intern end + alias _0_float _0_list + alias _0_symbol _0_list + def _1_list(*format) @format = format.join(" ") end + alias _1_symbol _1_list +} + +there are bugs with abstractions that use [route]->[list] instead of [route2] because they fail when $1="list". + +svn propset svn:mime-type text/pure-data *.pd + +Benchmarks on a Pentium M 1733 (with #out x11): +(fps) 0.9.0 0.9.3 0.9.4 +waves.pd rain 41.1 49.5 50.5 +game_of_life.pd 162 183 188 +sand.pd 18.7 20.5 20.1 +cellular_1d.pd 115 130.1 130 +wave_1d.pd 63.0 81.7 81.7 + +Benchmarks on a Pentium M 1733 (without window): +(fps) 0.9.0 0.9.3 0.9.4 +waves.pd rain 50.1 63.0 +game_of_life.pd 397 507 +sand.pd 27 30 +cellular_1d.pd 475 730 +wave_1d.pd 107 173 + + + |