path: root/externals/gridflow/TODO

/* $Id: TODO,v 1.2 2006-03-15 04:48:05 matju Exp $ */

[ ] SWIG: try -fcompact -fvirtual

ClaudiusMaximus matju: [#draw_image] inlet 0 method "op <opname>" is undocumented, but very useful :)
ClaudiusMaximus matju: [unix_time] is maldocumented, outlet 0 is a grid Dim[28](uint8), not a symbol


[ ] pd_examples -> examples
[ ] update doc
[ ] 0.8.1
[ ] make good use of the R type (automatic Ruby/C++ convs)
[ ] ditch xml, switch to docstrings or whatever
[ ] ...
claude wants http://en.wikipedia.org/wiki/Graham_scan

X Error of failed request:  BadValue (integer parameter out of range for operation)
error=0x2
request=0x91 (MIT-SHM)
minor=0x3 (X_ShmPutImage)
value in failed request: 0x04b008 == 240*320*4+8 (matju)

invalid read of size 1 (main.c.fcs:438)
malloc alignment = 4 mod 8 (en utilisant valgrind)

ClaudiusMaximus matju: feature request:  [#store4], behaves like store but accepts grids 
of floats as coordinates and interpolate using 4-point interpolation, eg, right inlet -> 
(3 3 f # 0 0 0 1 1 1 2 4 8), left inlet -> (1 # 0.5) outputs (0.5 0.5 0.5) 
(or whatever the interpolation would really be), left inlet (2 # 1.5 1.5) outputs (3)

AMD64 diffs: long 4->8, void * 4->8

<ClaudiusMaximus> hmm, sending a [reassign( message to the right inlet of a [#store] segfaults Pd
<ClaudiusMaximus> hmmm, i get a segfault with a simple patch sending "reassign, put_at ( 0 0 ), reassign" to the right inlet of [#store ( 3 3 # 0 )]
<alx1> matju: for the ./configure, I often get an error from aalib since common installs do not put the headers
<alx1> matju: would it be possible to make it a user defined choice like --with-aalib and turn it off by default?
<alx1> matju: not noticing that configure returns an error leads to an error when compiling
<alx1> back to the [display] object, it lingers for a moment after deletion
<alx1> and returns a 'pd_activate' error after I move it
<alx1> matju: once I delete it, unlike the other gui objects, it takes about one second before it dissapears, maybe a difference in tcl versions?

http://oprofile.sourceforge.net/about/
for 0.8.0:

drag_rectangle.pd is fucked (x11 error)
FObject#send_out2 : bself is NULL, rself=4097cbe8
[gf] ruby: ObjectSpace._id2ref(0x4097cbe8/2)
[gf] returns: #<GridFlow::FormatX11 [FormatX11 out]>

[ ] FIX LIBQUICKTIME CODE: #include <lqt/*> instead of <quicktime/*>
[ ] try removing or modifying -falign on PPC/OSX
[ ] c++ typecasting is nuts (see class Pt<T>)
[ ] make c++ refcount and ruby gc to cooperate.
[ ] encoding of quicktime RGBA only works with [#cast b].
[ ] add classmethod support to source filter
[ ] document the source filter
[ ] make example on separable convkernels
[ ] add concepts of hot inlet and conf inlet at the code level
[ ] [display] should use clocks to update tk less often
[ ] [display] leaks in server ?
[ ] optional args don't work in source_filter.rb
[ ] deprecation warnings
[ ] Add a video latency patch (in examples)
[ ] can't have FPatcher and GridObject behaviour at once ??
[ ] Numop: warn on misalignment
[ ] wrap Dim (?)
[ ] wrap GridInlet (?)
[ ] wrap GridOutlet (?)
[ ] wrap Grid (?)
[ ] wrap Numop
[ ] remove GridInlet* ?
[ ] configure chokes if GCC3 doesn't have the C++ plugin
[ ] [#change]
[ ] [#import~]
[ ] [#export~]
[ ] [#expr]
[ ] 8-align STACK_ARRAY and ARRAY_NEW ?
[ ] #: option swap
[ ] consider including [mysql] [renamefile] [ls] [shell]
[ ] re-enable int64,float64
[ ] added #matrix_solve, #slice
[ ] Copy-on-Write (COW)
[ ] < 10 bugs in buglist
[ ] Fully hyperlinked manual
[ ] internals documentation
[ ] internal grid protocol redesigned
[ ] raise limit of in/outlets from 10 to 100 or unlimited.
[ ] unbork make test and plain ruby mode.
[ ] add support for "anything" methods (def _0_(sel,*a))
[ ] look into -ftree-vectorize
[ ] should STACK_ARRAY remain at all?

for 0.8.1:

 * wrap Pd API completely.
 * test.rb should become much more serious. (the @pack bug really sucked)

Bugs (High Priority) (!!! Fix all of these before 0.8.0 !!!)

pick a number: [149]

[147] [#store] blows up in nervous_video (reassign)
[146] [#peephole] resize problem
[145] "list -1"->[@import] only tells error message once every other time
[144] small leak remaining in x11. may have to do with non-use of XDestroyImage.
[141] shouldn't be able to call Pd from a non-main Ruby thread.
[140] is [@convolve] actually cross-convolution?
	that is, convolves with a mirror image of the convolution matrix.
[139] [#peephole]
	<mis> matju: BUG-ALERT - peephole does not delete until a screen refresh
	<mis> matju: I was just about to do that
	<mis> matju: it works... almost
	<mis> matju: if you try to move the GOPed abstraction the peepholes don't move along
	<mis> he :)  better yet, the abstraction's bounding box's lower right corner will not move up or to the left of the low right corner of the peephole it contains.  Instead it will resize itself as you move it around.  
[138] [#peephole] crash
[137] [gf] peeperr: problem with set_geometry, wrong size.
[136] fix C_INCLUDE_PATH vs CPLUS_INCLUDE_PATH
[134] #import with zero size crashes?
[133] numop ** can freeze GF
[129] jmax2pd: [display] should not be translated
[128] jmax2pd(?): loadbang-messages (comma) don't get translated to .pd ??
[118] error messages can be unclear (eg. file not open... is it #in's, or #out's?)
[117] osx: [#store] crash instead of raising
[112] Pt<T>: some casts aren't done properly
[107] MMX computations may be done misaligned, which may be bad.
[105] when file not found, mpeg error messages are wrong, they say the filename is "".
[102] osx: error: shmget failed (cannot allocate memory) in X11, for regular pictures
[100] osx: libgridflow.so installs in wrong directory (of course)
[090] doesn't check whether ruby/version.h matches .so / .a
[027] tcp test is broken
[089] is LINK TO="..." not properly handled? see part of install.xml that uses it.
[088] FIX MOULINETTE.RB ! <attr name type default>
[078] "unsupported codec" bogus errors.
[097] bin/backtrace does not detect scripts
[005] can't open "": Success; loading a nonquicktime file using quicktime
[006] #in#open problem with String argument.
[010] separate bridge makefiles from other makefiles!!!
[011] research bugs in nonblock IO with Ruby. (eg. serial port)
[025] format_grid: write test for other bpv's
[026] latency is not that good. (GC ?)
[030] FormatGrid: not properly tested (crashes sometimes)
[031] stdout should be redirected if bridge is present
[032] error handling sometimes leads to crash
[033] error handling shows inanely useless messages (sometimes)
[034] format/mpeg3.c,quicktime.c might cause lockup on exit
[038] some seeds aren't typechecked
[039] jpeg file cat crashes on second file (jpeglib buffering?)
[040] GRID_FINISH triggered while backstore ???

Bugs (Low Priority)
[079] can trap segfaults in codecs of libquicktime.
[090] http://poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC30
[076] EINTR not handled properly in a lot of code?
[041] what would be a way to make GF work well with Pd's idea of a list?
[043] TCP server socket busy even after kill
[045] write unit tests about required functionality
[046] write unit tests about error checking
[047] make system to ensure everything is working before a release.
[048] potential issues between fork() and X11 ?
[051] remove the crap given by -w of ruby
[054] it's possible to crash by sending data in two inlets of a same object at once
[059] mess with options in format grid
[060] look for "!@#$" in source
[029] #join on small chunks is way too slow
[009] T_DATA #dup #clone ???

--- make the development process more test-driven:
    * file every feature request and bug report into base/test.rb so
      that gridflow fails those tests;
    * THEN make gridflow pass those tests.

===================

Community
[ ] put new examples on website

Distribution
[ ] should now plan which features are pre-1.0 and which are post-1.0
[ ] review all documentation
[ ] review all examples
[ ] make CDs

Installation
[ ] make the installer help the user more:
      * .h was found
      * compiled with .h correctly
      * .so|.a was found
      * linked with .so|.a correctly
[ ] installable doc
[ ] installable includes ?
[ ] produce linux binaries ("static")
[ ] produce cygwin32 or mingw32 binaries
[ ] produce macosx binaries
[ ] produce linux rpm/deb
[ ] less compile-time checks and more run-time checks
[ ] should GridFlow be bundled with some of its own dependencies?
[ ] reintroduce --puredata-dist-dir ?
[ ] --prefix (pour ruby)
[ ] install vs site-install mess
[ ] bridge should check that the gridflow.so file matches.
[ ] make uninstall
[ ] have option for gprof
[ ] cross-compile:
	http://www.libsdl.org/extras/win32/cross/
	http://rooster.stanford.edu/~ben/linux/cross.php

Documentation
[ ] make a short feature list as a promo
[ ] add credits
[ ] indiquer comment faire un rapport à propos d'une carte v4l ou d'un mode x11 non-supporté
[ ] project_policy.xml
[ ] specify policy for <help>, <icon>, etc.
[ ] things you type verbatim are all in KBD blocks
[ ] placeholders for your own values are all in I blocks
[ ] document our own release procedures, how our CVS works
[ ] document how to report bugs, basic troubleshooting
[ ] document our distribution of the archives
[ ] ascii conventions: [foo] vs "foo", etc.
[ ] document internals
[ ] describe indexmap in doc
[ ] document backward compatibility issues
[ ] typical uses: <typical></typical>
[ ] architecture: class->type/shape->use->role
[ ] it should be indicated whether an object is supported by pd/ruby/c++
[ ] heavily hyperlink documentation
[ ] automate more parts of the documentation
[ ] benchmarks (especially: simple video i/o)
[ ] mimic [pdp_help_input] [pdp_help_output]: they are convenient
--- remember to keep doc/help/screenshots/examples up to date
[ ] <jtm_> the suggestion would be to add a link at the end of each section leading to the next section

Source Filter
[ ] add \ruby \end blocks (multiline inlined rubyscripts)
[ ] \attr

Architecture
[0.8] <pat> aussi les erreurs que donne gf c'est beaucoup de ligne pour juste un file not found ;)
[0.8] reentrancy of objects (to allow true recursion)
[0.8] inlet-variables (variables associated to inlets)
[ ] eliminate the Dim class, replacing by Grid itself (?).
[ ] pull mode (functional-style)
[ ] bidirectional messages (constraint/relational-style)
[ ] send-reply messages (imperative-style)
[ ] add reflection (#global:classes, #global:methods, ...)
[ ] use MétaRuby typing to make things friendlier.
[ ] transmitting grids by diffs?
[0.8] dispatch on _n_-prefixed methods (one selector any inlet)
	  and     _#{i}_-named methods (any selector one inlet)
[0.8] generalize the backstore
[ ] Outlet#buffer should harmonise with the lcm of its corresponding Inlet#factor's.
[ ] lazy grids: Grid#operator Pt<T> makes sure all the grid is received
	and Grid#range(int start, int n, T bogus) makes sure the
	specified part of the grid is received.
[0.8] accounting: grids messages packets numbers bytes copybytes
[0.8] memory profiling
[0.8] compare Ruby API with both flext/pyext
[ ] add a "query" class for debugging other objects?
    (would be able to extract extra info from each object/class)
[ ] add a GridFlow::LaterException or something
    (which tells to retry later with the same packet because another
    grid reception is not completed yet.) 
[ ] interpret nested lists as n-dimensional.
[0.8] MMX memcpy() ?
[0.8] MMX _runtime_ autodetection

Operators
[ ] User-defined operators; compound operators... e.g. [# {{* 7} + {>> 3}}]
[ ] n-ary operators... make [#] variable-inlet
[ ] binary ops to replace unaries : rand -> ???, sqrt -> ???
[ ] trinary ops? eg. */ *>> *+ ?: minmax &<< >>&
[ ] numop2: rol, ror
[ ] numop1: weight (in bits)

Bridge
[ ] UnknownAtom class
[ ] Pd bridge should support list-literals inside messages
[ ] GFBridge object visible from Ruby

Computation Objects
[ ] #counter (similar to #for)
[ ] #inner inlet 2 -> 1, and other incompatibilities i wanted to introduce
[ ] #for "step" argument should have default {1...}
[ ] zero option in #convolve (?)
[ ] #convolve2 (simpler)
[ ] #convolve rewritten as combination of #convolve2+#border ?
[ ] #convolve(2): should allow n-D convolution
[ ] #complex_sq is backwards
[ ] #store: option zero (?)
[ ] #import~, #export~
[ ] #fold, #scan: option reverse (right-to-left)
[ ] #join: option swap
[ ] #matrix_inverse, #matrix_solve
[ ] #reverse (on given dimension number)
[ ] #timeline (or rtmetro): sequencing of events from grid
[ ] #replace_if, #random_select, #markov_chain, #unpack, #sort, #slice, #splice
[ ] #polygon_perimeter, #polygon_area, #draw_polygon smoothly
[ ] fuzzy #store (interpolate)
[ ] write a complete benchmark script
[ ] spatial biquads (ask Tom)
[ ] string operations to compensate for the feebleness of PD in that area.
	this includes [#sprintf].
[ ] #pack should become type-conscious (cast option, like #import)
--- see also APL/J/NArray/PDL/Aplus/Yorick/IPOW reference

Effects
[ ] #mosaic
[ ] #gaussian_blur, #sobel, #pixelize, #despeckle (median)
[ ] #dither, #draw_lines, #draw_points, #fire, #border, #queue
[ ] #chroma_key: i0=RGBpicture i1=RGBcolor i2=tolerance o0=RGBApicture
        with fuzzy flag; (don't call it #chroma_key though)
	maybe #mask...
[ ] turtle (logo style)
[ ] effectv
[ ] create abstractions for usage patterns (eg: like @remap_image)
[ ] see gimp for ideas
[ ] kaleidoscopes
[ ] FFTW
[ ] bilinear interpolation a+(b-a)*x+(c-a)*y+(a-b-c+d)*x*y (#scale_by smoothly?)
[ ] #scale_by/#downscale_by fractional values
[ ] bicubic interpolation
[ ] multilinear interpolation (n-dimensional?)
[ ] #daltonism
[ ] #perlin ?

Input/Output
[ ] replace "open <driver> <source> <filename>" by "open <driver> <mrl>"
[ ] videodev: is it possible to reach 60 fps ?
[ ] a #store that is persistent (saved in .pd) ?
[ ] Grid: support storage in all six types
[ ] Grid: bpv 1,2,4 uint
[ ] Grid: text version of the format
[ ] Gimp, GStreamer, ffmpeg, xine, mplayer, GEM, PDP, OSC
[ ] RubyX11 0.6
[ ] support for output inside of a GUI app (Tk/Gtk/Qt/...)
[ ] get subtitling of tv channels
[ ] add messages "load", "save" as open,bang,close combos. (?)
[ ] use 1.8.0's IO#read(Integer,String)->String
[ ] SDL: should support multiple windows
[ ] x11,sdl,aalib: use buffering to synchronise frames at a constant rate.
[ ] x11: sync with the actual vsync of the monitor
[ ] x11: maybe vram-to-vram copy using X11's Pixmap remote-objects
[ ] shell object?
[ ] #import, 'grid tcp': option: sudden vs gradual transmission through gridflow.
[ ] HPGL, [unixtime]
[ ] [parallel_port], [serial_port], [usb_mouse]

Format
[ ] remaining number type support in formats
[ ] all: possibly get rid of the "format" notion ?
[ ] PNM: P1,P2,P3,P4,P5 support
[ ] XBM, XPM, GIF support
[ ] fork: pseudoformat for launching format handler as separate process
[ ] put_at/reassign in x11/sdl/quartz
[ ] /dev/dsp, /dev/mixer
[ ] Generic SHM/MMAP Drivers
[ ] "open grid pipe" and such.
[ ] TCP: non-blocking write
[ ] TCP: buffer n grids
[ ] multi: multi-file (by glob pattern, e.g.: foo*.ppm)
[ ] AVI
[ ] format grid: "headerless until 10" (text files)

User Interface
[ ] color picker
[ ] gridbox (akin to intbox) or grid edition dialog, etc.
[ ] peephole:
    [ ] works without x11 (for osx...)
    [ ] does it really work with something other than 3 channels ?
    [ ] add a popup menu that makes sense
[ ] make ruby/tcl evaluator boxes retractable
[ ] vector slider (n parallel sliders in one, Dim(n))
[ ] 2d slider
[ ] radial slider (angle chooser)
[ ] drag-and-drop palette of object construction
[ ] object construction menu specialized for each class
	(generated from class reflection/metadata)

Extra/Examples/Etc
[ ] make demo app(s) in Ruby/Tk: bin/playmovie
[ ] .pd: read; write better
[ ] examples/generique.pd
[ ] add examples from the workshop, etc

how about creating aliases like "open window" = "open x11 here" (or arbitrary
user-defined sequence of messages settable in ~/.gridflow_startup)  and "open
videodev" -> "open videodev /dev/video0, option channel 1, option size 240 320" 
maybe there should be a separate ~/.gridflow_config that would be saveable from
gridflow, and ~/.gridflow_startup would be edited only by the user but
~/.gridflow_config would still be a text file.  i say that
because the startup is a ruby script so gridflow couldn't just do
search-and-replace in there, because the option could be inside a "if"
expression and such... at most, gridflow could _append_ text to
.gridflow_startup, to be non-intrusive...

Feature Requests

From: Dave Bergevin <dabergevin@eisa.com> 
  [ ] synchronized audio with the video when using mpeg1 & 2
  [ ] full screen mode (without the menu bars or frames)
From: Alexandre Castonguay <acastonguay@artengine.ca>
  [ ] server side pd to handle multiple incoming commands, audio + video streams
  [ ] support for audio in the mpeg streams

<matju> alx:
 le tab completion ne donnerait pas seulement les noms possibles pour le
prochain argument dans le objectbox; il donnerait aussi une description de
l'argument (surtout dans le cas où l'argument a beaucoup de valeurs
possibles) 
 je pense que le help de Pd est une bonne idée (le concept hands-on)
mais ça pourrait aller plus loin. 
 j'ai pensé intégrer le manuel de référence de gridflow directement dans
le programme, disponible au runtime, dans l'éventualité d'un GUI qui
saurait en tirer avantage. sinon on pourrait faire des hacks cool genre un
message "help" au inlet 0 enverrait une description de la classe dans le log.
 (sauf que dans ce cas: [rubyprint] est tenu d'afficher directement tout
ce qu'il reçoit. alors doit-il écrire "help" ou écrire le help ? 

alx1->matju: détection de plusieurs personnes; détection avec deux caméras
(?);  détecter les formes; détecter les couleurs; support mpeg4; hotspots

> http://redshift.sourceforge.net/debugging-GC/ [ruby-talk:53835]
> You might also want to search the web for "malloc debugger".

... I am Jack's Overwritten Stack Pointer...

<ircbridge> <GPS> Tcl_GetObjResult Tcl_GetErrorInfo ...
<ircbridge> <GPS> Tcl_GetStringResult may also be helpful.  (note I can't
seem to find Tcl_GetErrorInfo (maybe I dreamed it up)) 
<ircbridge> <GPS> matju: BTW CatchObjCmd might help too
<ircbridge> <GPS> I see that the new Tcl_Dict commands have taken over the HEAD's CatchObjCmd

ClaudiusMaximus reports that allocating a 197M grid causes a crash (turns out C++
throws exception, can't allocate, signal ABRT)

ClaudiusMaximus says: I want to be able to send output from Pd+Gridflow to a video4linux
loopback device so that other v4l programs can access the stream.
I found the video loopback device here: http://motion.sourceforge.net/vloopback/

<alx2> matju: iSight on linux :
http://episteme.arstechnica.com/6/ubb.x?a=tpc&s=50009562&f=8300945231&m=40110681
<alx2> matju: apparently it needs something called 'coriander'...I'm checking into it
<alx2> matju: the person who got it working mentions libdc1394
<alx2> matju: ah coriander is a GUI for firewire
<alx2> matju: on this page :
http://www.tele.ucl.ac.be/PEOPLE/DOUXCHAMPS/ieee1394/cameras/index.html
<alx2> matju: I learn that the iSight is compliant with version 1.30 of IIDC
<alx2> matju: the docs for coriander mention that the author needed to create a major and minor device for the camera.
<alx2> matju: retyping (damn pdf's) -> there are two devices to create: raw1394 and video1394
<alx2> matju: raw1394 has major 171 and minor 0 while video1394 had major 171 and minor 16
<matju> thanks
<alx2> matju: any useful info?
<matju> i'll see, doing something else at the moment
<alx2> matju: having a camera like the iSight supported could be fantastic for video capture/detection on a laptop.  Not to mention that the bttv cards are getting harder to locate

-mfpmath=sse2

<MiS> <matju> alx2: btw you cannot set helpfiles explicitly on _abstractions_... 
<MiS> yes you can
<MiS> and specify -helppath to add a path to help.  So selecting 'Help'
      in the pop-up on an abstraction will open a help fiel for that abstraction

"Bangs in a geiger counter have a l*dt probability of occuring
 in a dt time interval. Therefore the time till the next event
 follows a Exp(l) law, for which:
 f(t) = l*exp(-l*t)
 F(t) = integral of l*exp(-l*t)*dt = 1-exp(-l*t)
 F^-1(p) = -log(1-p)/l
 //return (uint64)(1000LL*ms*-log(1-drand()));"
matju 2002


remember to check for volatile VALUEs if GC problems ever occur

ruby -ve '[30,31,62,63].each {|x| printf "%d bits: %s\n",x,(1<<(x-1)).class }'

linx:
  http://cast.rubyforge.org (parsing C code)
  http://ruby-contract.rubyforge.org/doc/
  

sand.pd differential mode, 0.8:
fps   min   med   max   avg   stdev
17.57 54.82 55.81 78.00 56.89 3.93 : before
after ...