diff options
Diffstat (limited to 'README')
| -rw-r--r-- | README | 594 |
1 files changed, 0 insertions, 594 deletions
@@ -1,594 +0,0 @@ -$Id: README,v 1.3 2007-01-03 00:44:28 matju Exp $ - -PureUnity - -Copyright 2006 by Mathieu Bouchard <matju à artengine point ca> - -This program is free software; you can redistribute it and/or -modify it under the terms of the GNU General Public License -as published by the Free Software Foundation; either version 2 -of the License, or (at your option) any later version. - -See file ./COPYING for further informations on licensing terms. - -This program is distributed in the hope that it will be useful, -but WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -GNU General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program; if not, write to the Free Software -Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. - -+-+-+--+---+-----+--------+-------------+---------------------+ -GOALS - - 1. To provide a unit-test framework, which also provide benchmarking - features, all made in Pd for use in Pd. - - 2. To provide tests for functionality in internals, externals, abstractions, - etc., in a modularized way, in a DRY/OAOO fashion, thus abstracting out - common features so that many objects share the same test patch for the - features that they have in common. - -+-+-+--+---+-----+--------+-------------+---------------------+ -REQUIREMENTS - - 1. Pd 0.39 (PureMSP or Devel) - -+-+-+--+---+-----+--------+-------------+---------------------+ -TEST PROTOCOL - - new: - create common (reusable) fixtures. - - inlet 0: - bang: - run all available tests in that class. individual tests don't have - to be available through individual methods but may. If they do, the - names of the methods must match those given in the test results. - - each test should build its own non-reusable fixtures and reinitialize - common fixtures, not assuming that the previous tests have left the - common fixtures in a normal state. - - outlet 0: - test results. a sequence of lists like: - list $passed? $accuracy $elapsed $name1 ... - - where: - $passed? is either 0 for failure or 1 for success - $accuracy is a float proportional to relative error on math - (if not applicable, use 0) - $elapsed is a nonnegative float, the time elapsed in milliseconds - or it is any negative float meaning the time hasn't been measured. - $name1 and the rest are symbols and/or floats identifying the test - - for example: - list 1 0 -1 commutative f + * - - Which means that the 1st test about commutativity passed ($2=1) because it - was perfectly accurate ($3==0) and that we didn't measure the time ($4=-). - -+-+-+--+---+-----+--------+-------------+---------------------+ -SEVERITIES (in decreasing order) - - * crash: Segmentation Fault, Bus Error, Illegal Instruction, Infinite Loop, - etc. You can't deal with those errors at the level of the tests. Maybe there - should be a way to tell a test object to skip certain tests, by name, in - order to be able to perform as many tests as possible while waiting for a - fix. It could become possible to rescue from some of those crashes if Pd - supported exceptions (stack-unwinding). - - * corruption: this may cause future crashes and failures on innocent - objects/features. I have no solution for this except to be careful. - - * post(),error(),pd_error(): Gets printed in the console. The problem is that - those can't be handled by the test objects, so someone has to read them and - interpret them. Also they prevent test objects to ensure that error - conditions produce error messages. - - * pd_error2(): I wish this would exist. It would be sort of like pd_error() - but it would produce a pd message instead, whose selector would be an - error code, designed to be both localizable and [route]able. By default, that - message would be sent to the console, but there would be an internal class - designed to catch those messages. (If stack-unwinding were possible, it would - be disabled by default on pd_error2 and could be enabled explicitly - by-selector). - - * failure: a test object reports a problem through outlet 0. - - * dropout: a failure in realtimeness... difficult for an object to detect. - - * inaccuracy: a test more or less succeeds but the test detected that the - epsilon sucks. - -+-+-+--+---+-----+--------+-------------+---------------------+ -PROTOCOL FOR [error] - -new: - optional argument which would either be a float - (e.g. the $0 of the enclosing abstraction) or a pointer. - -inlet 0: - set $scapegoat: - replaces the originator of the message by $scapegoat, which can be a - float or a pointer - - error $1 ...: - causes its arguments to be concatenated, space-separated (may include - floats), and then sent through pd_error using the appropriate - originator (scapegoat). - - list $1 ...: - for future use. would use pd_error2() (see README or previous mail). - $1 has to be a symbol. - -+-+-+--+---+-----+--------+-------------+---------------------+ -ACCURACY AND ERROR (in math-related unit tests) - -The "absolute error" between a practical result and the expected value -is considered to be the distance between the two value. That is the -absolute value of the difference. - -In the case of positions in 2D, 3D, etc., use the L2-Norm which is -a generalized Pythagoras' Theorem: dist^2 = x^2 + y^2 + z^2 + ... -A norm is a distance between something and zero. - -Sometimes you have several practical results for one expected value -and must extract a single absolute error out of that. Then you should pick -the largest of the individual absolute errors. - -Sometimes you don't have an expected value, you just have several -practical results that you expect to be quite the same. In that case, -the absolute error is the "diameter" of those results. The meaning -of diameter here is: the largest distance between any two results. - -If in a single test you must compare 2D errors with 3D errors and 1D -errors, etc., you may have to adjust them by dividing the error by -the square root of N (N is the number of dimensions). In that case, -the resulting value is called a RMS (Root-Mean-Square). - -The maximum error introduced by just representing a number as a float -(instead of an exact value) is at most proportional to the magnitude -of the number (e.g. usually 16 million times smaller: about 6 decimals). -Also, often we are only interested in relative error, which is absolute -error divided by the norm of the expected result, because small absolute -errors don't matter much with large results. This is the reason floats -exist in the first place. By default, use relative error as the $accuracy -in Pd tests. - -If you don't have an expected result, then compute the relative error as -being the absolute error divided by the norm of the average of practical -results. - -In the RMS case of relative error, the norms of expected results should also -be adjusted, but both adjustments cancel because they get divided by each -other. That means: don't divide by the sqrt(N) at all and you'll get an -appropriate result. - -+-+-+--+---+-----+--------+-------------+---------------------+ -TYPE PREFIXES - -Those have to be prefixes in order to be honored by DOLLSYM: -[$1norm] should expand to [fnorm], [lfnorm], [#norm], etc. - -Those prefixes are necessary in order to achieve polymorphism through -abstraction arguments. - -CURRENT: - f float - ~ signal - -FUTURE (from PureData): - s symbol - p gpointer - a anything - l list (of whatever) - lf list of floats - ls list of symbols - lp list of pointers - -FUTURE (from DesireData): - t stringpointer - L listpointer - v varpointer (instance symbol) - -FUTURE (from GridFlow): - # grid (of whatever) - #b grid of bytes (uint8) - #s grid of shorts (int16) - #i grid of ints (int32) - #l grid of longs (int64) - #f grid of floats (float32) - #d grid of doubles (float64) - #r grid of rubies (VALUE*) - -for a type prefix to be considered implemented, it has to -have the following class set: - - metaabstraction for floats for signals for grids - [$1.inlet] [inlet] [inlet~] [inlet] - [$1.outlet] [outlet] [outlet~] [outlet] - [$1.do $2 $3] [$2 $3] [$2~ $3] [# $2 $3] - [$1.taa] [t a a] noop [t a a] - [$1.swap] [swap] noop TODO - [$1.norm] [abs] [env~] [# sq]->[#ravel]->[#fold +]->[#export]->[sqrt] - [$1.packunpack3] pack,unpack noop TODO - -The first two cannot be implemented as abstractions and instead must be -defined as aliases in pureunity.c. - -extra metaabstractions: - [$1.rand] [f.rand] [~.rand]TODO [#.rand]TODO - -+-+-+--+---+-----+--------+-------------+---------------------+ -OTHER PROTOCOLS - -Those four classes are operators that give verify algebraic properties -of other operators. The more their outputs are close to zero, the more -those other operators are faithful to an algebraic property. - -(here, supported $types are f and ~) - -[commutator $type $class] (2 inlets) ab-ba -[associator $type $class] (2 inlets) (ab)c-a(bc) -[distributor $type $class1 $class2] (3 inlets) a&(b^c)-(a&b^a&c) -[invertor $type $class1 $class2] (2 inlets) ab/b-a - -+-+-+--+---+-----+--------+-------------+---------------------+ -TESTS AND RULES - -For each class, a test file's name is the class name followed by "-test.pd", -and a rule file's name is the class name followed by "-rule.pd", -in the same way as it is for help files. - -for a class called $foo, the protocol (aka interface aka rule) $foo is the -set of behaviours expected from the $foo class; the class called $foo-rule -must repect the $foo protocol as well, plus it should test that the inputs -are valid, and if they are, it should test for one or several results and -report any errors. - -(((To report errors and inaccuracies, output them through the properties outlet at the right. If there is no -properties outlet in $foo (curently almost nothing in Pd has one), -then $foo-rule must have one more outlet than $foo.))) - -(((Float messages coming out of the properties outlet of $foo-rule report -accuracy. Named error messages come out with selector "error" followed by -an error-symbol and then its arguments.))) - -(((In the case of true/false logic, a value of 0 means that a test has passed -and a 1 means that a test has failed. Those values represent failure and not -success. The reason is so that it matches with accuracy levels, where 0 is -perfectly accurate, but any inaccuracy shows up as a relative error fraction. -Any finite nonnegative value is allowed for accuracy, because it is expected -to be the result of a norm))). - -(((In standard math, "Discrete Metric" is when there are only two possible -distances between objects: together=0 and apart=1))) - -+-+-+--+---+-----+--------+-------------+---------------------+ -RANDOMIZERS (?) - -+-+-+--+---+-----+--------+-------------+---------------------+ -ETC - - -(write me!) - - - - -If +-test.pd tests [+], it can test for hotness, coldness, it can test -that only one result is produced per hot message, that all results are -float, that a few example additions work, and that with random inputs it -respects commutativity, associativity, invertibility, within appropriate -relative-error bounds, etc. - -However +-test.pd can't test that errormessages aren't printed during the -testing. This may be something that we want to check for, and currently -the best way to handle it is to search the console for error messages, and -if there are any, restart the tests in verbose mode and see where the -error happens exactly. - -[...] - -Floating-point is the scientific notation for numbers that we all -learned on paper in school. Rounding and inaccuracy are two sides -of the same coin. They are required when it is stupid to have perfect -results, that is, when it would mean too many computations for little -gain. - -However sometimes we want to make sure that our math is accurate enough. -Many algorithms are data-recursive: each computation uses previous -results. Many of those algorithms have chaotic and/or unstable -behaviours, which means that the inaccuracies may skyrocket instead of -fading out. - -+-+-+--+---+-----+--------+-------------+---------------------+ - -Date: Fri, 13 Jan 2006 04:07:59 +0900 -From: Mathieu Bouchard <matju@artengine.ca> -Reply-To: ruby-core@ruby-lang.org -To: ruby-core@ruby-lang.org -Subject: Re: Design contracts and refactoring (was Re: mathn: ugly warnings) - -On Fri, 13 Jan 2006, mathew wrote: - -> *Dean Wampler *<deanwampler gmail.com> writes: -> > Let me suggest an XP-style alternative; make thorough unit tests -> > required and make sure they "document" - and test! - the design -> > "contract". -> Unit tests are not an alternative. They are an additional requirement. - -I find unit-tests to be often decomposable like this. Start with something -like this: - - raise if Blah.new(666) != Blah.new(666) - raise if Blah.new(747) != Blah.new(747) - raise if Blah.new(242) != Blah.new(242) - raise if Blah.new(69) != Blah.new(69) - raise if Blah.new(37) != Blah.new(37) - -then generalize it ("equality is defined based on the arg of .new"): - - for x in [666,747,242,69,37] do - raise if Blah.new(x) != Blah.new(x) - end - -then extract a contract from it: - - class CheckedBlah < Blah - def self.new(x) - r = super(x) - raise if r != super(x) - r - end - end - -so now all Blah object creation may be checked throughout actual uses of a -program and not just unit tests. The unit test now reduces to: - - for x in [666,747,242,69,37] do Blah.new(x) end - -so for many unit tests, all you have to do is just do things and discard -the results, and the contract will do the job of checking. - - _ _ __ ___ _____ ________ _____________ _____________________ ... -| Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju -| Freelance Digital Arts Engineer, Montréal QC Canada - -+-+-+--+---+-----+--------+-------------+---------------------+ - -Date: Fri, 13 Jan 2006 05:05:19 +0900 -From: Mathieu Bouchard <matju@artengine.ca> -Reply-To: ruby-core@ruby-lang.org -To: ruby-core@ruby-lang.org -Subject: Re: Design contracts and refactoring (was Re: mathn: ugly warnings) - -On Fri, 13 Jan 2006, mathew wrote: - -> For example, consider a simple vector addition routine in a 3D library. -> The unit tests might test its behavior with Float and Integer vectors, -> since that's why it was written. - -Here's another way to factor unit-tests that I haven't mentioned in the -last mail. - -suppose you test for + using: - - class IntegerTest - def test; 2+2==4 or raise; end - end - class FloatTest - def test; 2.0+2.0==4.0 or raise; end - end - class RationalTest - def test; Rational(2,1)+Rational(2,1)==Rational(4,1) or raise; end - end - -you can refactor those tests like this: - - class NumericTest - def initialize nt; @nt; end - def make x; raise "abstract class" end - def test; make(2)+make(2)==make(4) or raise; end - end - class IntegerTest; def make x; Integer(x) end end - class FloatTest; def make x; Float(x) end end - class RationalTest; def make x; Rational(x,1) end end - -> However, to do that you need to know whether the feature of supporting -> (say) Complex vectors or BigDecimal vectors is intended or not. The unit -> tests won't tell you this. - -[...] - -> > One limitation of documentation is that it has no enforcement power, -> > so you have to write tests anyway to test conformance. -> Unit tests have no enforcement power either, because you can just change the -> test. Indeed, I've already had to do this once when it turned out that the -> unit test was wrong. (In net/ftp.) - -That was a pretty bad case of strawman argument. Dean was assuming that -your documentation was not executable when you had quite clearly stated -that it was the contracts that acted as documentation! - -[...] - -+-+-+--+---+-----+--------+-------------+---------------------+ - -Date: Fri, 13 Jan 2006 07:36:36 +0900 -From: Mathieu Bouchard <matju@artengine.ca> -Reply-To: ruby-core@ruby-lang.org -To: ruby-core@ruby-lang.org -Subject: Re: Design contracts and refactoring (was Re: mathn: ugly warnings) - -On Fri, 13 Jan 2006, mathew wrote: - -> > The XP view is -> > that you should eliminate the redundancy. -> Except it's not redundancy. -> Unit tests define a set of functionality that is required. Documentation tells -> you the functionality that is supported, which is generally a superset of the -> functionality required by the unit tests. - -Let's follow the argument of both of you to the end. - -1. Unit-tests often match inputs with outputs on a case-by-case basis. - -2. Redundancy should be eliminated. - -(1) suggests that there is a shorter way to express the unit-tests. -Suppose you are able to find a formula for generating output-validators -from inputs. Then that formula is a postcondition of a contract, and the -explicit output-validators of the unit-tests are redundant. - -(2) because part of the unit-tests are redundant, part of the unit-tests -should be eliminated. This causes the postconditions to become an -essential part of unit-testing. - -Unit-tests vs contracts is a false debate. - - _ _ __ ___ _____ ________ _____________ _____________________ ... -| Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju -| Freelance Digital Arts Engineer, Montréal QC Canada - - -+-+-+--+---+-----+--------+-------------+---------------------+ -Date: Fri, 13 Jan 2006 17:19:41 +0900 -From: Mathieu Bouchard <matju@artengine.ca> -Reply-To: ruby-core@ruby-lang.org -To: ruby-core@ruby-lang.org -Subject: Re: Design contracts and refactoring (was Re: mathn: ugly warnings) - -[...] - -In order to entrench the tests-as-documentation habit firmly in the Ruby -community, we need a catchy acronym. Like RTFUT = Read the Fabulous Unit -Tests! - -+-+-+--+---+-----+--------+-------------+---------------------+ -http://lists.puredata.info/pipermail/pd-dev/2006-01/005920.html -Date: Fri, 20 Jan 2006 23:52:22 -0500 (EST) -From: Mathieu Bouchard <matju@artengine.ca> -To: pd-dev <pd-dev@iem.at> -Subject: macros and such (was: pd-lib, SIMD) - -[...] - -I think that the Pd source doesn't use nearly enough macros or other -code-reducing tricks. - -The reduction of code isn't so much about making things use less RAM: the -RAM excuse is quickly evaporating as even the tiniest computers come with -plenty of RAM and even the faster kinds of RAM come in ever more copious -amounts (big caches). - -The reduction of code is programmer-oriented. I'm not talking about length -of identifiers here (this is a separate issue). Every line of code should -do something interesting by itself. Code should read like a good story and -not like a car. Ever tried to read a car? It's boring. The same damn -piston copy-pasted 12 times. - -The reduction of code is also documentation-oriented. Once the programmer -has been contaminated with the wisdom required to make small code or -understand small code, then why wouldn't the programmer explain it to his -students in higher-level terms instead of chanting 12 times the same -piston as if it were a marathon of Hail-Marys ? - -This is why Pd needs a taxonomy of object classes. If I don't get that -taxonomy in Pd itself nor in its help files, at least I'll have it in its -unit tests. - -Once and only once. -Once and only once. -Once and only once. -Three strikes and you refactor. -for x in [1,2,3] say: Once and only once - -http://c2.com/cgi/wiki/?ThreeStrikesAndYouRefactor - -BTW I'm not talking about only inheritance of implementations. The most -important thing to me is inheritance of expectations, so that if I name -100 classes that obey the rule "Operator2", then you have just learned -something common about 100 classes. - -Operator2 means right-inlet is cold, left-inlet is hot, there is a "set" -method for using left-inlet as cold, there is a "bang" for explicitly -activating the main computation. The main computation only produces one -message. That's what "Operator2" means in my taxonomy, and it's that much -that hasn't to be stated explicitly in each help patch. - -Help patches can be abstractions to be used to by other help patches. Just -put a [operator2-help] object in your help patch to indicate that the -currently documented class obeys the standard operator2 rules. - -Who's against it? - -+-+-+--+---+-----+--------+-------------+---------------------+ -http://lists.puredata.info/pipermail/pd-list/2006-02/035169.html -Date: Sat Feb 4 21:22:29 CET 2006 -From: Mathieu Bouchard <matju@artengine.ca> -To: pd-list - - * Previous message: [PD] dealing with arguments and inlets - * Next message: [PD] Re: [PD-announce] A new version of FFTease is now available for Pd - * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] - -On Fri, 3 Feb 2006, Hans-Christoph Steiner wrote: - -> The way I have been thinking is that the first inlet is the general -> inlet, and it can accept many types of messages. Then the second inlet -> lines up with the first argument, the third inlet to the second -> argument, etc. - -I agree. Many objects obey the rule that the k'th inlet matches argument -$k for several arguments in a row, usually all of them. - -> I think this is pretty clean and flexible, and I think -> it would be nice to have some kind of standard for this. - -And the best way to make sure people are following a standard is to make -it so easy to follow that it's harder to not follow it than to follow it. -Of course I don't mean adding hurdles to doing it otherwise, but rather -make a shortcut for those who follow the standard. Short of this, people -who make abstractions/externals can get a friendly reminder, from someone -who cares, that it would be better if they followed the standard. - -> Obviously, it doesn't work for all objects, but I think it would be good to -> standardize on objects it does work for. - -PureUnity's goal (when I work on it) is to design a taxonomy that -separates objects that obey certain properties, from those that don't, -because that's a way to reuse tests, but also because certainly it doesn't -hurt documentation either, and it's even better if it can influence how -abstractions are made. - - _ _ __ ___ _____ ________ _____________ _____________________ ... -| Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju -| Freelance Digital Arts Engineer, Montréal QC Canada - -+-+-+--+---+-----+--------+-------------+---------------------+ -From matju@artengine.ca on Dec 18, 2006 - -I thought up some kind of classification of type systems, avoiding to call -them strong/weak or static/dynamic because those words are confusing. - -1. Typed expressions: each piece of code that can give a value, has a -type that can be figured out at compile-time. - -2. Typed variables/parameters: declarations allow runtime checks but not -compile-time checks. - -3. Typed values: variables don't have types, they can contain any value, -but every value has a type. - -4. Typed uses: values don't have types, a type is a way of using a value. - -Strictness, in the sense of forbidding things to the user, is not on that -scale, it's another aspect. A well-balanced strictness allows one to -bypass the system whenever needed, but without being too error-prone. - -However it's difficult to say what it means to "bypass the system" for all -four typing categories at once, or even within one category. |