# My Application to the Holman Prize, or How You Can Help Me Help You with STEM Data Visualization Accessibility

There's something I've needed to do for a while.

Libaudioverse is a massive digital signal processing project, and it is the nature of digital signal processing that much of the debugging process would go much more easily if I could examine data. But I can't because I'm blind and the tools to let blind people do so do not exist.

All we have at the moment are graphing calculators. Input an equation, output a graph. This is nearly useless for anything but high school mathematics. The real world is not made up of homework polynomials, nor are you finding where it crosses the y axis over and over.

Fixing it is time consuming. My plan was to take Libaudioverse as far as it can go without, then write the bare minimum for my personal use. This may have been useful to others, but it wouldn't have done much outside the specific domain of DSP. Also, documentation? What documentation?

## So What's the Plan?

First, I'll get a mailing list up and facilitate discussion. I know that I need to solve problems for digital signal processing, machine learning, and other general programmer sorts of things. Programming isn't the only science out there, and I'm not the only person who wishes tools existed. What I want to do is get as many use cases as possible, then go through them and find the commonalities.

While that's happening, I solve the following core problems that will show up in every domain:

• No good library exists for soundscapes and/or sonification. We need a number of small sonifiers that can be linked into bigger constructs. Examples include lines, one-off events, clicks, etc. all designed to be fed mathematical data.

• No tool exists to graph arrays of ordered pairs of real numbers. This will look a lot like the current graphing calculators, except that it's designed to be used as a library and fed data from scientific simulation scripts.

• No tool exists that can handle complex numbers. As far as I am aware, no one has tried to solve this one. I have some ideas, but a sonification is worth a thousand words in this case.

• No tool exists to work with scalar functions and/or data of two variables (that is things such as f(x, y) = xy). This will probably involve a UI that allows one to "feel" it using a touchscreen; there may be a way to do it without one, but I have not yet had a workable idea in this regard.

• No tool exists to monitor realtime changes in parameters, i.e. measuring how much different layers in a neural network change.

• While reading off lists of numbers technically works, no tools exist to allow blind people to work out trends or relationships when dealing with discrete data such as bar and pie graphs, scatter plots, etc.

• This all needs to be integrated to some degree with current science stacks and given a workable UI, while still maintaining the ability to output to other places such as wave files on disk whereever possible.

As I solve these, I'll write tutorials and make example audio. It isn't enough to have tools; people need to know how to use them effectively. It would be nice if general knowledge gained by the sighted about different sorts of visualizations transferred, but we're completely in uncharted waters here. I suspect they will be very different, in the same way that screen readers are significantly different from a mouse.

Finally, throughout this process and with whatever time and money is left over, I'll incorporate whatever came up on the mailing list and/or in other discussions, starting with whichever ideas seem most likely to benefit the largest number of people. I am certain that what I am discussing can be done within the \$25000 budget, but it is the nature of software development and research that it's hard to say for certain how much will be left at the end.

Then, a whole bunch of blind people who otherwise couldn't, can.

## But Why Me? Why not Someone Else?

If you're still reading then you may be asking yourself why I think I'm qualified. It's easy for me to prove I'm a good programmer; I've got a number of rather large pull requests against the Rust compiler, Libaudioverse, etc. I was able to bootstrap my DSP knowledge to the point of writing a library for audio synthesis on my own as a blind person by learning LaTeX and finding one sort-of-accessible textbook. When I think I can do a project, i'm almost always right.

But that's not actually my most important qualification. My most important qualification is that I am blind and need the tools.

One thing that comes up over and over in my discussions with sighted people is this: in much the same way that a blind person isn't going to understand color schemes or fonts or good UI layout or what-have-you, a sighted person doesn't understand the world of the blind. You'd never ask a blind person to develop the next matplotlib. For much the same reason, I would never ask a sighted person to develop these tools. Someone who can only academically understand isn't going to make something that works well. And we really need this to work well, especially since getting the funding to redo something is harder than getting the funding to do it in the first place.

If you're still here and you haven't, signal boost this. Winning the Holman Prize will give me the means to make a difference to a whole lot of people.