Szymon Kaliski

Improving Replit's Console and a Database-y Creative Coding Environment

Hi, happy New Year!

Tinyletter had an "incredible journey" moment recently, so if you're subscribed, you're receiving this through Buttondown — more expensive than the free Tinyletter was, which hopefully is a good thing for not having to migrate any time soon again. If you relied on the archive, you can always read the past editions on my website.

I spent the last quarter working on improving Replit's Console, and noodling on a toy database-y creative coding environment, more below!

Improving Replit's Console

I redesigned and rebuilt Replit's Console (almost) from scratch. A single terminal view was replaced with a virtually scrolling list of terminals, where each execution happens in its own context, is reliably cleaned up, even if the process misbehaves (thanks to the magic of cgroups), and which is already starting to integrate more elements of the IDE into this single pane.

Check out this blog post for the reasoning behind the project, and various implementation details.

Throughout this work, two things were often on my mind:

• One, how the line between frontend and backend doesn't really make sense, and how the only line that matters is between the user and the machine. A lot of "UI" and "UX" in this project stem from deep backend work, and wouldn't be possible to achieve by only redesigning the frontend bits.
• Two, how so many details are important to get right (and how often a detail actually ends up being a big engineering effort) for the tool to "disappear in your hand". For example, the behavior of blocks growing with their content up to the point where they take over the whole pane, and the gesture of scrolling up "just works" where in reality, the logic is pretty complex — which reminds me a bit of the infamous "how Slack decides to send you a notification" diagram.

Database-y Creative Coding Environment

Outside of work, I've been noodling on a new creative coding environment, in which you generate images by querying the existing state, and asserting a new one, in a sort of transactional way, with the hope that each of these transactions will also have a meaningful representation, so seeing the intermediate steps requires no additional work, it is the work.

Let's see how we could create something resembling the Space Colonization algorithm in the tool.

First, we assert that there's a page to draw on, place a bunch of attractors, and a single node in the middle:

Second, we give each of the nodes two "zones" — one which "growth" is directed by, and another, smaller one, where the attractors are "consumed". So far so good:

Third, we have to grow towards the attractors by creating new nodes, and consume them if they are in the consumption zones:

This big step does a lot, all at once, which doesn't really help much with understanding what's going on. There's iteration, there's a lot of state, and a lot of playing computer in your head.

To make it worse, we also loop this step 20 times, to let the structure grow out a bit (with a simple number picker out of the frame), and you can only see the last step of that process.

Finally, we can drop all the circles, leaving just the lines...

...which we can export, convert to G-code and plot on the DIY Pen Plotter, which was my plan all along:

Let's go back to the middle step — as you probably realize, it is the one that I'm pretty unhappy with. It feels as if I've been struggling with some version of this problem for a while now — representing and visualizing single values is not that hard, but as soon as collections come into play, the simple solutions fall apart.

We hit this earlier on in Inkbase where interacting with a single item was nicely visualized with inspector panes, but groups of objects were not. We tried to explicitly solve this in Crosscut, but the solutions were far from ideal. I also hit this in my research at Glide where displaying lists of values is the main thing you do.

So far, I've seen two interesting approaches to solving this:

• The spread operator in Apparatus, which is well illustrated by the Bar Chart example — notice how the X property spreads between 0 and 10. It is a single property, but represents a collection, and each of the other operations "just work" (you can sin(X) and get back a spread of values — I've heard that this is pretty much a "List Monad").
• Engraft by Josh (and earlier Pane) both have map behavior as part of the core interaction, with each step being visible and represented on the screen. Engraft is especially promising — an attempt at building an environment where various live, visual, domain-specific tools can coexist and communicate with each other.

One more random thought — the idea of "rubbing a database" on a creative coding library could be an interesting tangent to explore on its own. For example, imagine having computed facts that could be queried too, like intersections, nearness, etc. — which feels similar to how Realtalk works (a database-OS powering Dynamicland).

Anders Hoff a.k.a. Inconvergent has also been exploring home-grown Datalog for making computer art, which is worth checking out (both for the computing, and the art).

Worth Checking Out

What I've been reading lately:

• I circled back to some readings on Embodied Cognition (all great!):
  • Being There
  • Supersizing the Mind
  • Cognition in the Wild
• I started a thread on properly (re-)learning mathematics with A Programmer's Introduction to Mathematics — I felt for a long time that I might not have paid enough attention in school to some important connections, and while I do know some things here and there, I don't have a proper "structure" on which to hang them; I accept that this will be a longer learning project, and so far it actually feels quite nice to be going through proofs and exercises.

On the web:

• A cool live multimedia spreadsheet for programming
• Origami crease patterns
• Annotated lambda diagrams
• Elliot is at it again
• The Gulf Between Design And Engineering
• a bunch of great UI ideas in this new DAW — especially around spatial queries, and the mechanics that allow one to derive a delay effect from interactions between send and return blocks
• a really nice write-up on Spellburst and UIST 2023

Fin

Let me know if you have any thoughts about making loops Live, Rich, and Composable, or about the intersection of databases and creative coding.

Best wishes for 2024!

Backlinks

1. 2023-12-03Liunon — Database-y Creative Coding Environment¹