stat.lsp

Module index


Module: stat.lsp

Basic statistics and plotting library

Version: 2.0 - fixed plot an plotXY routines for gnuplot
Version: 2.1 - doc changes
Version: 2.2 - frequency vector in stat:power was shifted left by 1
Version: 2.3 - $TEMP should be global
Version: 2.4 - changed deprecated name to term
Version: 2.5 - fix in cov-matrix and dependents multiple-reg and corr-matrix
Author: Lutz Mueller, 2001-2010

Functions for statistics and plotting with GNU plot

To use this module it has to be loaded at the beginning of the program file:
 (load (append (env "NEWLISPDIR") "/modules/stat.lsp"))
 ; or shorter
 (module "stat.lsp")
 
All functions work on integers and floats or a mix of both. lists are normal LISP lists. matrices are lists of lists, one list for each row in the two dimensional data matrix. See the function stat:matrix on how to make matrices from lists.

This file also shows how to use some of the built in matrix math functions like multiply, transpose, invert and fft.

Note, that the plot functions need gnuplot installed, a free graphing package available for most operating systems. See: http://www.gnuplot.info/

If no Gnuplot program is found by the module all non-plot routines are still usable.



Summary of functions


Plot functions (requires gnuplot)

 stat:plot     - plots one ore more vectors (lists)
 stat:plotXY   - plots an XY graph from to vectors
 
The plot functions rely on a $HOME/tmp directory on Mac OS X and other Unix or c:\tmp or c:\tmp on Windows. When loading the module, a message will be printed to stdout if either the gnuplot executable or a temporary directory cannot be found.

The plot functions are designed for interactive use with no special options, titles on the top or the X and Y axes. Only a legend is printed in the top right corner to identify different data ranges. On Win32 plot functions should only be used when using newLISP in a command shell, not inside the newLISP-GS application.

To produce more refined graphics the file tmp/plot can be edited to produce a different output. The Gnuplot command-file plot will look for data files in the same directory. One file for each data range.

General uni- and bi- variate statistics

 stat:sum      - returns the sum of a vector of numbers
 stat:mean     - returns the arithmetik mean of a vector of numbers
 stat:var      - returns the estimated variance of numbers in a vector sample
 stat:sdev     - returns the estimated standard deviation of numbers in a vector
 stat:sum-sq   - returns the sum of squares of a data vector
 stat:sum-xy   - returns the sum of products of a two data vectors
 stat:corr     - returns the correlation coefficient between two vectors
 stat:cov      - returns the covariance of two number vectors
 stat:sum-d2   - returns the sum of squared differences of a vector from its mean
 stat:sum-d2xy - returns sum of squared diffferences of two vectors
 stat:reqression - calculates the intecept and slope of a regression estimate
 stat:fit      - return the fitted line using regression coefficients
 stat:f-prob   - return the probability of an F-ratio of two variances
 stat:moments  - calulates 1st to 3rd moments from a vector of numbers
 

Multi variate statistics

 stat:multiple-reg  - calculates a multiple regression
 stat:cov-matrix    - calculates a covariance matrix
 stat:corr-matrix   - calculates a correlation matrix
 

Time series

 stat:smooth   - smoothes a vector of numbers
 stat:lag      - calcultes a difference list with specified lag
 stat:cumulate - cumulate a data vector
 stat:power    - calculate the power spectrum of a time series
 

Matrix and list utilities

 stat:matrix       - make a matrix from column vectors
 stat:diagonal     - make a diagonal matrix
 stat:get-diagonal - return the diagonal of a matrix in a vector
 stat:mat-map      - map a binary function on to matrices
 


§

stat:corr

syntax: (stat:corr X Y)
parameter: X - A list of numbers.
parameter: Y - A list of numbers.

return: Correlation coefficient of lists X and Y.



§

stat:cov

syntax: (stat:cov X Y)
parameter: X - A list of numbers.
parameter: Y - A list of numbers.

return: Covariance of data in lists X and Y



§

stat:cov-matrix

syntax: (stat:cov-matrix X)
parameter: X - A list of numbers.

return: Covariance matrix of X with N rows and k columns.



§

stat:corr-matrix

syntax: (stat:corr-matrix X)
parameter: X - A list of numbers.

return: Correlation matrix of X with N rows and k columns.



§

stat:cumulate

syntax: (stat:cumulate X)
parameter: X - A list of numbers.

return: The cumulated list of X.



§

stat:diagonal

syntax: (stat:diagonal item N)
parameter: item - The diagonal element.

return: A diagonal matrix of length N with item in the diagonal.



§

stat:fit

syntax: (stat:fit X Y)
parameter: X - A list of numbers.
parameter: Y - A list of numbers.

return: fitted line based on (stat:regression X Y).



§

stat:f-prob

syntax: (stat:f-prob F df1 df2)
parameter: F - The variance ratio.
parameter: df1 - Degrees of freedom.
parameter: df2 - Degrees of freedom.

return: Probablity of F variance ratio for df1, df2 degress of freedom.



§

stat:get-diagonal

syntax: (stat:get-diagonal X)
parameter: X - An matrix filled with numbers.

return: A list from the diagonal elements of X.



§

stat:lag

syntax: (stat:lag X n)
parameter: X - A list of numbers.
parameter: n - Lag n.

return: A differenced list of X with a lag of n.

If the length of list X is l then the length of the resulting differenced list is l - n.

§

stat:mat-map

syntax: (stat:mat-map op A B)

return: Matrix map, e.g. (stat:mat-map + A B).

Used for adding and subtracting matrices.

§

stat:matrix

syntax: (stat:matrix C1 .... CN)
parameter: C1 - The first column list of values.
parameter: CN - The Nth column list of values.

return: A matrix off 1 to N columns C.



§

stat:mean

syntax: (stat:mean X)
parameter: X - A list of numbers.

return: The mean of data in list X.



§

stat:moments

syntax: (stat:moments X)
parameter: X - A list of numbers.

return: Calculates all moments of list X.



§

stat:multiple-reg

syntax: (stat:multiple-reg X offY)
parameter: X - A list of numbers.
parameter: offY - Zero based offset into Y.

return: Multiple regression of vars in X onto Y at offsetY.



§

stat:plot

syntax: (stat:plot p1 p2 ... pN)
parameter: p1 - First list of data.
parameter: p2 - Second list of data.
parameter: pN - Nth list o data

return: The process ID of the gnuplot process.

example:
    (stat:plot '(1 2 3 4 5 4 3 2 1))

    (set 'data-A '(1 2 3 2 1 2 3))
    (set 'data-B '(2 2 2 2 2 2 2))
    (stat:plot data-A data-B)
The first example plots one data range. It will appear as series-1 in the legend. In the second example two data ranges are drawn each in a distinct color the variable names data-X and data-Y will be used as labeld in the legend.

§

stat:plotXY

syntax: (stat:plotXY list-X list-Y [srt-style])
parameter: list-X - List of x-coordinates to plot.
parameter: list-Y - List of y-coordinates to plot.
parameter: str-style - Optional style parameter, e.g. "line"

return: The process ID of the gnuplot process.

example:
    (set 'data-X (random 1 10 100))
    (set 'data-Y (random 2 8 100))
    (stat:plotXY data-X data-Y)
Data points X,Y are plotted on the plane. If no style is specyfied points appear as little crosses. The style "line" would connect all data points.

§

stat:power

syntax: (stat:power TS)
parameter: TS - A time series of numbers.

return: The power spectrum of a time series



§

stat:regression

syntax: (stat:regression X Y)
parameter: X - A list of numbers.
parameter: Y - A list of numbers.
returns (b0 b1) coefficients of regression Y = b0 + b1*X.

§

stat:sdev

syntax: (stat:sdev X)
parameter: X - A list of numbers.

return: Standard deviation of data in list X.



§

stat:smooth

syntax: (stat:smooth X alpha)
parameter: X - A list of numbers.
parameter: alpha - Smoothing coefficient 0 < alpha < 1.

return: Exponentially smoothed sequence in X.



§

stat:sum

syntax: (stat:sum X)
parameter: X - A list of numbers,

return: Sum of data in list X.



§

stat:sum-d2

syntax: (stat:sum-d2 X)
parameter: X - A list of numbers.

return: Sum of squared diffs (x - mean(X))^2 in list X.



§

stat:sum-d2xy

syntax: (stat:sum-d2xy X Y)

return: Sum of squared differences (x - y)^2 of elements in lists X and Y.



§

stat:sum-sq

syntax: (stat:sum-sq X)
parameter: X - A list of numbers.

return: Sum of x*x data elements in list X.



§

stat:sum-xy

syntax: (stat:sum-xy X Y)
parameter: X - A list of numbers.
parameter: Y - A list of numbers.

return: Sum of products x*y data elements in lists X and Y.



§

stat:var

syntax: (stat:var X)
parameter: X - A list of numbers.

return: The variance of the data in list X.





- ∂ -

generated with newLISP  and newLISPdoc