Szymon Kaliski

Solving Things Visually

• two main things here:

  1. visualization (charting) as a way to see and understand the data
  2. solving things by means of drawing/sketching - diagrammatic/geometric thinking

• for the first one:

  • visualization is a macroscope - allows us to work with/look at complex ideas: (source: Model Driven Management of Complex Systems ↗)
  • we have a lot of tools for creating them in software, but no "silver bullet"
  • I have a feeling that there are important primitive interactions/UIs missing for creating a generic data-visualization-building software

• for the second one:

  • Bret Victor makes a point that programming is currently blindly manipulating symbols - visualization is a solution for this blindness

    the author sees and manipulates indirect symbolic representations, and must imagine how they give rise to dynamic behavior

  • diagrammatic/geometric thinking seems to be an important skill in problem-solving, a cognitive style (after Seymour Papert) which allows us to work with the ideas visually, usually by sketching
    • Dynamic Medium could be great for thinking this way - if only we had proper tools!

• 3Blue1Brown's ↗ YouTube math explainers are very visual, and seem to give great intuitions into mathematics because of that

  • can I imagine a tool for directly manipulating them into existence?
    • Chalktalk ↗ by Ken Perlin seems to be one such imagining (with actual implementation)
    • Apparatus ↗ by Toby Schachman is another one (later productised into Cuttle ↗)
  • would such imaginary tools be good only for transferring knowledge, or also for gaining new knowledge? - in other words would I invent calculus just by having access to a Dynamic Medium that works this way?

Point of View Is Worth 80 IQ Points

— Alan Kay

• we can provide new point-of-views in software through (data) visualization

So far, I haven't actually made much use of intersections, but it's likely that I'm not yet "thinking geometrically" enough.

— Drawing Dynamic Visualizations Addendum ↗ - Bret Victor

• Bret mentions "thinking geometrically" a bunch of times in this article, I think it's an important insight of feedback loops between tools we use, and styles of thinking that they promote (Tools Building Us)

Sanjay had a neat trick of using GraphViz to debug his compiler by outputting and laying out various compiler graphs (e.g. CFGs). It is obvious when you think about, but a normal compiler dev wouldn't think to do that.

— https://twitter.com/seanmcdirmid18/status/1225852873096876032 ↗

Visibility is everything. In many cases the most effective person isn't the smartest or even the most qualified, it's the person who has visibility into the problem. Learning to make things visible gives you a superpower.

— https://twitter.com/ibdknox/status/1225860283744174080 ↗

Debugging may simply involve carefully watching for an unexpected change

— Aesthetics of Computation - Unveiling the Visual Machine

• if we have perfectly visual systems, we can "See Everything" and just watch for what's going on, and notice when things don't work
• another very similar point:

  There is really no boundary between the code and the world, and this allows for some wonderful workflows. For example, there is no boundary between programming and debugging - if something is acting weird, you can simply pause time, open the logic, and probe widgets and wires to see what state they're in. Tweak the logic on the fly and resume again. It's blissfully iterative and doesn't require you to learn any additional tools, e.g. a time-travel debugger or a Whitebox.

  — I did Advent of Code on a PlayStation ↗

  • again, this is not about "VPL" which is node-and-wire interface, but about a VPL which:
    • shows its state to you - the values flowing through the system
    • is "live" (as in Live-Programming) and allows you to prod at the values to see what happens

A crucial feature of the differential analyzer was that it dealt with mathematics as graphics. Dealing with mathematics in terms of pictures was perfect for applied physics and science, Bush wrote, as these fields typically dealt "with functions as a whole" and resorted to numbers only "as a rather laborious means of dealing with functions or the curves which represent them."

— Building SimCity - Chaim Gingold