The problem statement is:
Given a set of points, in 2D, compute the closest pair of points.

The brute force algorithm takes time. There’s a better solution described at the Wikipedia page Closest pair of points problem, which takes time.
As far as I know, there aren’t any posted solutions using OpenCL to compute the closest pair. So I’ve implemented one, it’s posted at closest-pair.

There are a few challenges in adapting the algorithm for OpenCL. Namely, we can’t use recursion so we must convert the recursive algorithm to a procedural one. In this case it’s not to complicated because the structure of the solution is easily converted from the top-down recursive algorithm to a bottom-up parallel algorithm. There are some tricky issues when the number of points are not a power of 2. I’ve commented the code for those cases.

The second challenge is using WebCL. WebCL has an additional restriction that you can’t pass structures between Javascript and OpenCL kernels. Because of this I had to use dumb arrays of simple uint’s instead of using arrays of uint2, uint3, and uint4. This made the code more verbose. To help reduce the verbosity I added macros in the OpenCL ckStrips kernel. The WebCL version is posted at
closest_pair_ocl.html.

I hope the solution is useful to someone. Enjoy reading the code, it requires careful thought.
Check it out at closest-pair or view the WebCL one online at closest_pair_ocl.html.

The steps on Ubuntu (or any other modern Linux with Emacs 24) are

1. Enable the http://marmalade-repo.org/ elpa package archive by adding the below to your .emacs file and restarting Emacs

       (require 'package)
       (add-to-list 'package-archives 
       '("marmalade" .
         "http://marmalade-repo.org/packages/"))
       (package-initialize)
   

2. Install multi-web-mode by using “M-x package-list-packages” and scrolling down to “multi-web-mode”.
3. Add the below to the bottom of your .emacs file and restart Emacs

       (require 'multi-web-mode)
       (setq mweb-default-major-mode 'html-mode)
       (setq mweb-tags '((php-mode "<\\?php\\|<\\? \\|")
                         (js-mode "]*>" "")
                         (css-mode "]*>" "")
   	              (c-mode "]* +type=\"text/x-opencl\"[^>]*>" "")))
       (setq mweb-filename-extensions '("php" "htm" "html" "ctp" "phtml" "php4" "php5"))
       (multi-web-global-mode 1)
   

   The important part is the “c-mode” section that will enable C highlighting for OpenCL kernels in html files.

4. Start coding!

I’ve shamelessly stolen the algorithm from there and done a simple conversion to Javascript.

The code is under the canvas-geolib GitHub repository in the geometry.js file, there’s also an example @ enclosingdisk.html. The example initially consists of three points and the smallest disk enclosing them. Click anywhere on the canvas and a new point will be added and the disk redrawn.

I don’t want to go over the general algorithm but I do want explain computing the unique disk with three points given for its boundary. In geometry.js the function “enclosingDisk3Points” takes three points and returns the unique disk that has those points on its boundary.

The below figure shows the two defining characteristics of the disk, which are (1) the disk center is centered at (x,y) and (2) the distance from the center to all three points (p0, p1, & p3) is equal i.e. the distance from the center to all three points is d.

From this computing (x,y) and d is simple. For simplicity we assume and , also we let .

The set of equations to solve is:

, , and

Solving for y, we have , using this we can solve for x, which yields Finally we also have , which finishes our computation.

The below Javascript code implements the above computation and also adds the preprocessing steps of (1) translating the origin to and (2) rotating the coordinate system so that is of the form

// return the unique disk with p1, p2, and p3 as boundary points.
function enclosingDisk3Points(_p1, _p2, _p3){

    var p1 = [_p1[0], _p1[1]];
    var p2 = [_p2[0], _p2[1]];
    var p3 = [_p3[0], _p3[1]];
    if (dist(p1, p3) > dist(p1, p2)){
var p = p2;
p2 = p3;
p3 = p;
    }

    var p = p1;
    // make p1 the origin
    p2[0] = p2[0] - p1[0];
    p2[1] = p2[1] - p1[1];
    p3[0] = p3[0] - p1[0];
    p3[1] = p3[1] - p1[1];
    
    // apply rotation matrix to make p2.x = 0
    // the rotation matrix is
    // | p2[1]/dist(p2), -1 * p2[0]/dist(p2) |
    // | p2[0]/dist(p2), p2[1]/dist(p2) |
    //
    var original_p2 = [p2[0], p2[1]];
    p2[0] = 0;
    p2[1] = d(original_p2);

    // apply rotation matrix to p3
    var original_p3 = [p3[0], p3[1]]
    p3[0] = original_p2[1]/d(original_p2) * original_p3[0] - original_p2[0]/d(original_p2) * original_p3[1]
    p3[1] = original_p2[0]/d(original_p2) * original_p3[0] + original_p2[1]/d(original_p2) * original_p3[1]

    // the unique disk with the points p1, p2, and p3 as boundary points is
    // defined by the equation y = p2.y/2 & x = (d(p3)^2 + p3.y * p2.y)/(2 * p3.x)
    var y = p2[1]/2.0;
    var x = (d(p3) * d(p3) - p3[1] * p2[1])/(2 * p3[0]);

    // apply inverse of rotation matrix
    var rotated_x = original_p2[1]/d(original_p2) * x + original_p2[0]/d(original_p2) * y
    var rotated_y = -1 * original_p2[0]/d(original_p2) * x + original_p2[1]/d(original_p2) * y;

    // translate back
    rotated_x = rotated_x + p1[0];
    rotated_y = rotated_y + p1[1];
    
    var radius = d([rotated_x - p1[0], rotated_y - p1[1]]);
    return [[rotated_x, rotated_y], radius];
    
}

The implementation difficulties are from using floating point arithmetic, which is not robust for geometric operations. The JTS library attempts to minimize this by a coordinate snapping technique. But for the operations used in solving Tangrams the provided snapping was not sufficient.

There’s an option in JTS to specify the snapping tolerance (it has a fairly small default). I added small wrapper functions for the two operations of Boolean intersection and Boolean difference. The wrapper functions apply successively larger snapping tolerances up to a factor of epsilon, where epsilon = 1e-5. The below code shows the wrapper functions (in the code, g1 is the Tangram and g2 is a puzzle piece).

    public static Geometry SemiRobustGeoOp(Geometry g1, Geometry g2, int op) throws Exception {
        double e1 = EPSILON/10;
        double snapTolerance = GeometrySnapper.computeOverlaySnapTolerance(g1, g2);
        while (snapTolerance < e1) {
            try {
                Geometry[] geos = GeometrySnapper.snap(g1, g2, snapTolerance);
                switch (op) {                    
                case DIF_OP:   // difference
                    return geos[0].difference(geos[1]);
                case UNION_OP: // union
                    return geos[0].union(geos[1]);
                default:
                    throw new Exception("unhandled semirobustgeoop: " + op);
                }
            } catch (TopologyException e){
                snapTolerance *= 2;
            }
        }
        return null;
    }

    public static boolean SemiRobustGeoPred(Geometry g1, Geometry g2, int pred) throws Exception {
        double e1 = EPSILON/10;
        double snapTolerance = GeometrySnapper.computeOverlaySnapTolerance(g1, g2);
        while (snapTolerance < e1) {
            try {
                Geometry[] geos = GeometrySnapper.snap(g1, g2, snapTolerance);
                switch (pred) {                    
                case COVER_PRED: // 
                    return geos[0].covers(geos[1]);
                default:
                    throw new Exception("unhandled semirobustgeopred: " + pred);
                }
            } catch (TopologyException e){
                snapTolerance *= 2;
            }
        }
        return false;
    }

Using the wrapper functions was key to a more robust implementation. The below figure shows a solved Tangram puzzle (from Test.java:FitTest_ToSingleLargeTriangle()), in the figure the puzzle pieces are labeled l1.dat, l2.dat,…, l7.dat (I was lazy in naming the files). It’s the result of running FitTest_ToSingleLargeTriange() in Test.java and plotting the result using gnuplot.

I used the symmetry of each puzzle piece and a heuristic for choosing which puzzle piece to fit in reducing the number of permutations used for solving a Tangram.

For the puzzle piece symmetry, I used that the square is completely symmetric so only its first line segment needs to be used when fitting it. The triangle pieces are only partially symmetric so two of their three line segments need to be tried.

The below figure illustrates the symmetry of the square:

I used two heuristics for which pieces to try fitting, first try larger pieces before smaller ones and two skip a piece if another identical piece has already failed to be fitted (there are two identical small triangles and similarly two identical large triangles).

With the above two optimizations it takes ~1min to solve a Tangram. Without the optimizations the algorithm did not complete for the Tangrams I tried.

The goal of our work was to beat the current forecast methods used by the Department of Water Resources for the April through July American River flow. The Department of Water Resources uses an aggregation of human judgement and linear regression equations for generating their forecasts. Given their methods they are surprisingly hard to beat!

We spent a few months trying different Machine Learning methods with little success. Many of the methods we tried resulted in forecasts that were significantly worse than the current forecasts, a few methods such as a properly trained neural network gave forecasts that were comparable to the current forecasts. Finally, I decided to use a Support Vector Machine (SVM) for producing forecasts, after testing a large combination of parameters the forecasts started being significantly better than the current ones.

The data we used for generating forecasts is available online @ https://github.com/bjwbell/California-Water-Runoff-Forecasting. The takeaway message is that we improved the forecast relative error from ~65% to ~48%. The below table shows the forecasts for the last 10 years.

The forecasts currently used by the Department of Water Resources produced relative errors of 63.82% and root relative squared errors of 69.15%. Using modern methods for SVM’s gave us an increase in relative accuracy of over 15%! This was a fantastic result and shows the large payoff in keeping up with the state of art for something as ordinary as river flow forecasting.

The Art of R Programming is a straight forward explanation of R for programmers who are reasonably familiar with programming in another language. Matloff makes no assumptions of expertise in C or algorithms and his explanations are succinct and easy to follow.

If you’re aren’t familiar with R, it is a statistical programming language, with some similarities to Matlab.

Rating 9/10

The big advantages of R are (1) it’s high level, (2) reasonably easy to read, (3) functional in nature, (4) simple syntax. If you’re familiar with Python, it has a similar feel. Compared to complex languages such as C++, Java, etc, R is a breadth of fresh air due to the lightness of its syntax. That said as a programming language Python is nicer. R has a few annoyances (for me at least) that make it less pleasant to write in than Python.
A couple of those are:

• Non-standard assignment operator e.g. to assign 5 to x in R we use “x <- 5" instead of the normal "x = 5" used in other languages. This is annoying because a significant amount of programming is doing assignments and a two character assignment operator is twice as much typing. Contrast this with Python which uses the plain "x = 5".
• Vector creation using “c(1,2,3,4)”. Vectors in R are similar to lists in Python, it would be more natural to add a little syntactic sugar and use “[1,2,3,4]” for vector creation i.e. the same syntax as Python and many other languages.

The real reason to use R are its statistical libraries, it’s very widely used for statistics and is the most pleasant environment to work in.

The areas Matloff covers are:

Much of the material is available online in tutorials such as John Cook’s, R Language For Programmers. The real gems are the chapters “Writing Fast R Code”, “Interfacing R to Other Languages”, and “Parallel R”. These chapters have great information that is not easily discoverable otherwise.

“The Art of R Programming” is a fun read, albeit somewhat specialized. If you need to do statistical work as a programmer I highly recommend buying it and spending an afternoon browsing it.

Photos and slides are posted here and the slides are also viewable below.

Rating: 10/10

Miran Lipovaca has done a fantastic job of writing a book aimed at beginning Haskell programmers. I like his very straightforward writing style of introducing each topic with the minimum of complexity.

As you probably already know the book is available online for free at http://learnyouahaskell.com so the question becomes why purchase the dead tree copy? To be honest I don’t recommend buying a dead tree copy if you have a Kindle.

To summarize there are three ways to read the book:

• Dead tree
• EBook
• Online

My preference in order is the e-book on the Kindle, the dead tree version, and finally the online version. For some reason I don’t like the formatting of the online version as much as either the dead tree version or the ebook. It’s partly because I have a harder time reading books on the computer than either on the Kindle or in paper back.
No Starch did a great job of printing “Learn You a Haskell for Great Good!” on quality paper and the binding is excellent. As with all previous No Starch books I’ve purchased the physical layout and ink quality of the book are both excellent.

The table of contents is:

• 1. Introduction
• 2. Starting Out
• 3. Types and Typeclasses
• 4. Syntax in Functions
• 5. Recursion
• 6. Higher Order Functions
• 7. Modules
• 8. Making Our Own Types and Typeclasses
• 9. Input and Output
• 10. Functionally Solving Problems
• 11. Functors, Applicative Functors and Monoids
• 12. A Fistful of Monads
• 13. For a Few Monads More
• 14. Zippers

Miran does an amazing job of keeping each chapter short and interspersing lighthearted comments. The table of contents should be self explanatory, as you can see he covers mostly just the fundamentals of Haskell and ends with Monads. There is the extra chapter on zippers which seems like an afterthought.

Comparing this book to Real World Haskell, they serve slightly different purposes. “Real World Haskell” assumes that you are already reasonably proficient as a programmer whereas this book introduces each topic without assuming much programming background knowledge. Also “Learn You a Haskell for Great Good!” is meant less as a reference text and more for reading from start to end. They are both fantastic books and I highly recommend reading both of them. “Learn You a Haskell for Great Good!” is a gentler introduction to Haskell and I recommend reading it first.

Most of the following tips work equally well for using Emacs with other SQL databases besides MS SQL Server.

How to connect to SQL Server using Emacs

First you need to connect to a database. The command for connecting to a MS SQL Server instance is “sql-ms”.
Once you’ve entered the connection information Emacs opens a SQL query buffer, in which you can type SQL commands. After you type a SQL command, don’t forgot to enter “go” to execute the command.

Helpful commands for retrieving schema information

1. select * from sys.tables
Display the list of database tables

2. select left(column_name, 35), left(data_type, 15), is_nullable from information_schema.columns where table_name=’tablename’
Display the column names and column data-types for the table, ‘tablename’.

3. select left(CONSTRAINT_NAME, 100), CONSTRAINT_TYPE FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE table_name=’tablename’
Display the constraints for table, ‘tablename’.

4. select left(@@servername, 25), left(DB_NAME(), 25)
Display the database server name and the database name.

Improving Data Display

To improve line wrapping execute “M-x toggle-truncate-lines”, this prevents the soft line wrapping that Emacs normally does and improves the display of long lines.

To make the display of wide columns (such as varchar or char(8000)) readable use select left(columnName, 25) from table, this truncates the column width to 25 characters.

Useful Shortcuts

Because sql-ms and the other sql providers inherit from comint you can use all the nice emacs command line shortcuts. I use “C-c-l” to list the command history, “M-p” and “M-n” to cycle through commands, and “sql-send-buffer” (C-c-C-b) to send a buffer to the *SQL* mode buffer, this is extremely useful if I’m trying to tweak a large query. Use sql-set-sqli-buffer to set the SQL interactive buffer that’s used by sql-send-buffer.

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads Wow.

Crunchy numbers

The average container ship can carry about 4,500 containers. This blog was viewed about 18,000 times in 2010. If each view were a shipping container, your blog would have filled about 4 fully loaded ships.

In 2010, there were 16 new posts, growing the total archive of this blog to 74 posts.

The busiest day of the year was January 20th with 673 views. The most popular post that day was Writing A Compiler In Haskell (Compiler Series Part I).

Where did they come from?

The top referring sites in 2010 were reddit.com, en.wordpress.com, moodle.cs.ualberta.ca, blog.llvm.org, and mail.live.com.

Some visitors came searching, mostly for line segment intersection, writing a compiler in haskell, nginx asp.net, nginx mono, and mono nginx.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

Writing A Compiler In Haskell (Compiler Series Part I) January 2010
11 comments

Line Segment Intersection May 2009
6 comments

Having Fun with Happy (Compiler Series Part III) April 2010
4 comments

Configuring Mono and ASP.Net on Nginx February 2009
3 comments

Playing with Alex (Compiler Series Part II) January 2010
4 comments