Cool Project Ideas

Question

I like to have at least one side project going at all times and often get bored with one and would like to play with another. Sometimes I find myself (as now) unable to come up with an inspiring, quick side-project.

As I was thinking about it I thought it might be nice to have a community record of projects that people have thought up and thought about working on but never had the time to do. I don't really expect that anyone will look at this list and want to do these things exactly, but I do think it's likely that this would serve as good inspiration.

Also I can only imagine that it'll be cool to see what the people here think up to do.

Answer 1

Core Data Structures Package(s)

Background

Mathematica is basically a data-structure free zone. This sucks. Using data structures efficiently makes code cleaner, faster, easier to maintain, and quicker to develop. They're great.

Idea

It'd be great if there were a package with some core data-structures built out and supported, either via LibraryLink libraries or via high-level Mathematica coding, like I do for some of my packages. Ever since I moved to not using a bunch of List and Association args I was able to rapidly develop a bunch of new code off the data-structures I wrote because I didn't have to think about the nastiness of dealing with free-floating lists and ensuring good structure/behavior. It'd be fantastic to have lots of these that people could just drop into a package as a dependency.

Implementation Thoughts

I have this InterfaceObjects paclet that makes it easy to create an object in the mold of like SparseArray and friends. They're immutable, Association-based creatures that are designed to not memory-leak, cause memory-bloat when reused, or to have unexpected unpacking or things like that.

The InterfaceObjects package can be dropped into a BTools-style package as a dependency (I do this already for some of my stuff) and makes it pretty easy to rapidly make new objects as it handles much of the nastiness of type-checking, property management, methods, etc. I got at least four robust, highly-useful objects out of it in about one day of coding and I use them all over my packages.

I'm picturing this providing the basic object structure and then methods/properties being bolted onto this. In the cases where a LibraryLink library may be used, this would basically mean the constructor to the InterfaceObject creates a LibraryLink object and allows the InterfaceObject to hold this and provide methods on top of that. Where that's not possible, methods on the internal stored data (like a PackedArray) can be used to allow for methods and manipulation to easily be implemented in an efficient, modular way.

Answer 2

Language Server Protocol for WL

A Language Server Protocol is a way to write IDE and editor functionality (autocomplete, go to definition, etc) without a dependency on a specific editor or IDE.

I'd like to see this done because I normally use Emacs, and while I really like Wolfram Notebooks, I can't see myself ever writing a full-featured, 1000+ LOC package in one. I'd like to be able to use Emacs, or any other code-specific editor, to do that job.

Halirutan's IntelliJ plugin is a great starting point. The hope would be to pull many of the features out of that and into a separate LSP server. Then that functionality would be easily accessible from Emacs, Sublime Text, Vim, VSCode, and most other modern editors.

The specification for LSPs lives here. Effectively, an LSP server is a HTTP server that implements interfaces such as completion or documentLink.

I don't think it would be super easy to write this in WL itself, although it does seem like it might be possible (and entertaining) to do so. Otherwise, it might be easiest to write one in Java since the starting point, Halirutan's IntelliJ plugin, is in Java.

Answer 3

SE Package Creator

Background

There are tons of high-quality posts with functions that are package ready or nearly package ready here. I can only imagine it would be useful to have collections of these functions packaged up in a nice distributable format.

Idea

I think it'd be cool to use the SE API to pull these kinds of posts and their code blocks down for a set of filters (maybe user, topic, score, etc.) and make it easy for people to create a package directly from a collection of answers. Some kind of convenient palette or interface would help a lot.

Implementation Thoughts

My basic idea was to use the SE API connection that I have here to do this. It can do that filtering stuff. Then use some XMLElement parsing to get the code blocks or alternately build an XMLGraph to pull these bits using CSS selectors. After that it could basically spit answers into individual files that people can then edit to make their package, probably using the paclet template/setup structure that BTools provides. Not sure how all this would play out in practice, but it definitely would be helpful.

Answer 4

Bug Tracking Interface

Background / Idea

The best way to get the inspiration to quash a bug is to have that bug exposed in the open. It'd be great to have a nice way to manage one's own bugs and public bugs in a simple way that integrates nicely with GitHub and other tools. This can be a great tool for development as well, as this can serve the dual purpose of helping you keep track of your issues and keep track of features you'd like to put in too.

We could also use this kind of thing to track WRI's bugs in a clearer format, as then at least we'd have a public record to point to when things go sideways or we are wondering if the behavior we're seeing is a proper bug.

Implementation

I started creating a BugTracker package which provides a convenient notebook interface for bug tracking and ways to export this to a Dataset and things. This could be extended to automatically export bugs to GitHub via the Issues system as that way this can be much more readily seen. It can already export a list of issues to Markdown so that can be posted on the web.

This package would need to be a more user friendly and make it easier for other people to report bugs too. It'll also need a standard storage format that people can contribute too and some tools for keeping track of who submitted what and what its status is.

Answer 5

JuliaLink

Background / Idea

I don't know much about Julia, but I would like to. It seems to be an up-and-coming system and has a lot of nice features. It'd be nice to have something like J/Link for it.

Implementation

I set up a J/Link-like link to python which works quite nicely if you can handle the compilation difficulties. Since Julia seems to be easier to integrate with at the C++ level I'm imagining this would be much easier to do on the core library side and then I'd basically just need to mimic the J/Link setup. It wasn't bad to do in python at all by basically just translating and adapting the existing Java code.

Answer 6

General Language/Box Parser

Background

The FE currently can tokenize Mathematica code so that it can be handled via the "box language" (and thus also colored). Unfortunately this parsing mechanism isn't exposed at all. It'd be nice to have syntax coloring for other languages too, especially vis-a-via "ExternalLanguage" cells.

Idea

Basically, just build a framework for doing the necessary parsing and reconstruction into and from some kind of AST for an arbitrary language given a spec. This AST could then be transformed into the Box language that's necessary for the FE to do syntax coloring and such.

Implementation Thoughts

General types of AST builders are well studied and we'd only need to implement one in the kernel. We might even be able to leverage Leonid's work here to do that. Not sure how that tokenizes things. In terms of moving in and out of the FE through boxes that's pretty easy using ExportPacket and a RowBox setup. All types of syntax coloring could then be pretty cleanly configured once we have this.

Answer 7

IDE Stylesheet and Package

Background

The Mathematica FE could provide a nice lightweight FE for editing small/medium size projects.

Idea

By creating a "ProjectManager`" package with all the necessary project-management functions and a nice stylesheet for hooking into this we could create a little IDE inside Mathematica by providing a tree-view to the .wl and .m files inside a folder. This little IDE could support plugins and extensions so that people can easily build off of it

Implementation Thoughts

The "ProjectManager" package could hook into the project-management functions I already package up into my BTools package. All it would really need would be a Stylesheet that provides access to these functions via a DockedCell and a nice ProjectObject that exposes project details so that people can write extensions easily off of these projects.

The .m and .wl files can be loaded via NotebookOpen[..., Visible->False] and dumped into a main notebook pane. Tabbing can be done by simply keeping track of the current files open and putting these as little tab names under the main DockedCell pane. Saving can just dispatch to either NotebookSave for the regular .nb files or to a ExportPacket[..., "SaveRename"] for .m and .wl files.

Custom stylesheets might be a bit harder to support... but for simple IDE work we can just ignore that or save the other Notebook options when loading a tab and just export the notebook content. Or we can figure out a way to make this basically a DockedCell-only package where we don't even need a custom Stylesheet

Answer 8

Physics engine link

The goal is to have access to a physics engine inside Mathematica.

This could be done by building a link to a physics engine such as JBox2D. The link should make it possible to define, visualize, and run simulations using Mathematica graphics primitives.

Answer 9

Harnessing Leonid's Let in debugging/profiling/debugging/proving.

My entire code-base is broken down by Let to a very embarrassing degree. But an attached debugger/profiler/documenter/program prover would at least allow me to claim this as part of a master plan. The debugger could show the state of the entire program at any point in time, the profiler identify bottlenecks, the documenter help show expected expressional structures while some correctness proofs might even be possible (something likely to be enhanced by upcoming TemporalLogic functionality).

Answer 10

Implementation of the Abstract Algorithm

Background

I read a Medium post linked to by @RobertJacobson about how using a Graph-rewriting algorithm some functions in functional programming languages can obtain asymptotically better runtime. For those who can get behind the paywall, here's a link to the core paper (it's only like ~15 pages really).

Idea

Mathematica supports a functional programming core and a pre-made AST to work with. This part of the language could be made asymptotically faster in some cases using this algorithm.

Implementation Thoughts

We already have the AST for a Mathematica call (the expression). Then we have a very explicit algorithm defining the types of rewrites that will optimize. Hopefully, then, we can just write a few ReplaceRepeated rules that will take us there...

Here are some resources:

• Lecture explaining the algorithm
• Example of how this works
• JavaScript implementation

Answer 11

Robust Jupyter kernel based on Xeus

Xeus is a framework that makes it easy to implement Jupyter kernels in C++.

https://xeus.readthedocs.io/en/latest/

https://xeus.readthedocs.io/en/latest/kernel_implementation.html#implementing-the-interpreter

It could be used to create a robust Jupyter kernel for Mathematica. Xeus would handle the communication with Jupyter, and the Wolfram Engine kernel would be handled through MathLink (WSTP)

For an example of calling the Wolfram kernel from C, see factor.cxx.

---

Note that the Mathematica kernel is designed to communicate with front ends through MathLink. If we want a Jupyter kernel to provide a similar experience to what we get with Mathematica's own notebooks, this is the way to go.

Answer 12

A Smarter Context Menu

A few years back I developed a workable framework but assumed WRI would get around to creating an "industrial" version.

Answer 13

Mathematica as Document Writer

Background

I already have a package that will take my Mathematica notebook and automatically convert it into a Markdown file with the appropriate type bindings and conversions and things. It'd be nice to be able to have a framework to do this for any file format (e.g. HTML, TeX).

Idea

Create a general Cell-style based file transformation package where different formats may be easily plugged in by simply defining a basic set of DownValues for a new file format.

Implementation Thoughts

I have my NotebookToMarkdown package which provides a pretty comprehensive and robust framework for writing a .md file from a Notebook expression.

The basic framework can be generalized out and the bindings for Markdown can be made modular so that bindings for e.g. HTML can be subbed in instead.

This way we can reuse all of my work of creating a framework that is able to be smart about when it saves and how it handles what to save and what to change and what Cell styles to use and all that but just be able to customize the parts that would benefit from customization.