Claiming "DataFrame" to be two-dimensional almost caused me to discount the entire project. On the contrary, it seems like they are actually multi-dimensional (since they are essentially OLAP relations) and you should advertise them as such. That they can be viewed as two-dimensional tables is incidental.
Thinking of DataFrame as a 2D structure just aids mental visualization. With hierarchical indexing they can be arbitrarily highly-dimensional so maybe I should sell it a bit more like that. In an OLAP setting the "sparse" format may often be better than the "dense" (truly N-D) format.
It would be an interesting avenue to pursue building a "big data" on-disk OLAP engine with pandas-like semantics (e.g. expressing groupby operations with the same syntax but operating on big data on disk or across a cluster of computation nodes).
No, they are multidimensional even without hierarchical indexing. e.g.:
x y z value
0 0 0 32
0 0 1 64
0 1 0 23
0 1 1 3.14
1 0 0 4.3
etc.
This is essentially a 3D cube of data, no hierarchical indexing involved. The benefit of hierarchical indexing is that you can wrap your spatial dimensions into a single real dimension for e.g. code abstraction.
I have actually been developing a similar library for OCaml (even with hierarchical indexing). It is good to see our libraries share many of the same ideas! I wonder though, have you considered GPU acceleration? AFAIK neither Matlab nor R do this natively yet.
