Some context: D3.js (https://d3js.org) is a popular and influential JavaScript library for building data-driven visualizations. There are a zillion awesome examples of using it to create all variety of diagrams with animations, interactivity, and live data sources. Arguably, D3 doesn't know much about any specific type of visualization, but it's the toolkit to assemble whatever you want.
Observable Plot is a separate library that focuses on the higher level task of generating specific common plots and charts. Compare a bar chart in D3 (https://observablehq.com/@d3/bar-chart) with Observable Plot (https://observablehq.com/@observablehq/plot-bar). The former involves creating SVG elements, the latter is just a function call.
The APIs are worth studying because they've decomposed visualization into elemental components that can be recombined. Using D3 can be like writing React for the first time, requiring a new mindset.
Both libraries come from the same prolific author, Mike Bostock (@mbostock here), who previously worked on visualizations for the New York Times. Mike's new company, Observable (https://observablehq.com), a very thoughtful take on computational notebooks, is also worth a look.
I've been secretly hoping he'll take on Graphviz next.
This is such an excellent observation (about Graphviz); you can see some of the rough outlines of the primitives that might get the job done, and how they might be composed. This area is so ready for new work. There are also some problems that maybe don't arise the same way in stat graphics and charts for data journalism, like coping with nested layouts, stable dynamic layout, optimizing topological properties, and intricate combinatorial optimization problems, to name a few things.
Observablehq scratches such an itch for me. It's exactly what I wanted out of Jupyter, for the cells to be reactive. I just wish I could have it in a desktop app. I'd pay hundreds for that.
Hard agree! The reactivity is a killer feature. That it hasn’t come to jupyter is my largest criticism of that ecosystem and the developers behind it. It’s like they have never used excel and seen what it’s good at or something, or are too focused on competing with VS Code when thats a losing battle and they should focus on the actual notebook interface (with things like reactivity).
The new geo mark looks great, congrats on such awesome progress on the Plot library, Mike & Fil!
Visualizing any kind of data in a few lines of code quickly is a very compelling use case, something that had been pretty difficult with d3 and various charting libraries before — to the point where writing custom Canvas/SVG code instead somehow always felt easier for me. Now I'm really compelled to learn Plot :)
1) Do you have any thoughts or insights having now made both a low level graphics library (D3) and now a higher level library with “Plot”, on the nature and utilities of such tools it even data vis in general?
For instance, I remember when Plot first came out, I noticed many popular D3 users were enamoured with how quick and easy it was to use, to realise it seemed many of them might have been using exclusively D3 for so long, they didn’t really know about these sorts of tools that us “plebs” use (not that “Plot” didn’t look good, it did!). Having used both D3 and higher level tools (not Plot, yet, I haven’t been working in JS land lately), I felt like the nature of the technologies is rather discrete such that they’re different worlds and you kind of have to choose which one you’re in and stay there.
Which leads to my second question…
2) The documentation indicates that Plot is built on D3 (which makes sense!). Is there any capacity to manipulate a Plot graph with D3 once it has been constructed? I’m not sure what this would look like exactly, but blurring the lines between higher and lower level graphic manipulation would be interesting.
I like having multiple tools that make different design trade-offs. There’s no “one ring to rule them all.” For analysis I like to start with a high-level tool so that I can explore quickly and see if the data is interesting. I can then “eject” to a lower-level abstraction if I reach the limits of expressiveness in the higher-level tool and determine it’s worth the additional effort to drop down.
Observable Plot is optimized for fast visual exploration: it’s concise, with good defaults, and lots of built-in conveniences for transforming data. In this regard Plot is similar to Vega-Lite and other high-level tools based on the Grammar of Graphics. Whereas D3 is optimized for bespoke presentation, like the sort of work we were doing at The New York Times; D3 prioritizes expressiveness.
I recommend Plot for most people over D3 because it’s more efficient: more of your time can be spent deriving insight from your data rather than programming. But there are still plenty of things that Plot can’t do yet (without plugins), like brushing a scatterplot, panning and zooming, and animation. We’re working on those!
And yep, Plot is built on D3. It’s nice to not have to re-implement the low-level visualization primitives like scales and layouts. Plot generates SVG output, so you can readily manipulate what Plot generates using D3 selections. Also, Plot is highly extensible: you can implement custom mark types, custom symbols, custom transforms, etc. Here are two examples, the first to apply some custom clipping, and the second to design a custom symbol for a dot map:
We haven’t fully documented how to write custom marks in Plot yet, but we plan to do that (along with animation and interaction) prior to the 1.0 release. There’s a ton to do but it’s exciting and we love hearing feature requests!
Re 1), I like Mike's talk from the 10th anniversary of D3. It came while working on Plot but before it was officially released; a few of the points relate to the high/low-level thing and anticipate Plot:
Re 2), Plot returns a regular DOM node, so you can select any part of it and do anything you’d otherwise do with D3. Plots also expose their scales: assuming the DOM node of a generated Plot is named myPlot, you can call myPlot.scale("x"), and then feed that domain and range into a D3 scale. I can find some examples later if you’re interested.
Also, tangentially ... I just went on to https://bl.ocks.org/mbostock for old time's sake, knowing full well that Observable is your new home, and got a Heroku error. Presuming it's because the bl.ocks are no longer maintained, I figured I'd let you know.
The tooltip example in particular strikes me as a convincing demonstration that there's a fairly coherent vertical stack of tools in D3->Plot around which a useful and flexible ecosystem of plugins and extensions can be built, all in SVG too (of course!)
> You either don’t have permission to view this notebook, or it doesn’t exist.
Forgot to hit that share button? ;)
And hey, wasn't there research showing that the bad reputation of pie charts is kinda undeserved, as in that they don't actually perform any worse than bar charts when viewers were asked to estimate relative quantities? Meaning they're like the Comic Sans of data viz: their horribleness is vastly overrated because people abuse them
I gave a presentation once on why I kinda like pie charts but it was based on zero research so I'd be interested to see that. I mean I think if you're just showing one whole they're as intuitive as any visualization can get. It's a populist cause.
>What cyclic modular spatial visual encoding can you find to encode your proportions other than angle? I get that it’s sorta hard to compare across pie charts. But if you see a stacked bar chart, nothing about it signals that it’s out of 100%! No Cartesian encoding is a non-arbitrary 100%! Whereas pie charts need no axis! Everywhere in the universe, 360º ≅ 0º. Pie charts carry their axis with them throughout the universe! That’s very special!
From a data visualisation perspective, pie charts are kind of terrible. Primarily because they are really easy to misinterpret, and because they handle many divisions poorly. I prefer spark charts, like github's language percentage bar.
> Primarily because they are really easy to misinterpret
No, that's the thing. This is something that was asserted early one by someone influential, without any data to back it up. And then when HCI people started to actually test it they found no issues whatsoever.
I wish I had bookmarked the sites/videos where I saw this.
Observable Plot is a separate library that focuses on the higher level task of generating specific common plots and charts. Compare a bar chart in D3 (https://observablehq.com/@d3/bar-chart) with Observable Plot (https://observablehq.com/@observablehq/plot-bar). The former involves creating SVG elements, the latter is just a function call.
The APIs are worth studying because they've decomposed visualization into elemental components that can be recombined. Using D3 can be like writing React for the first time, requiring a new mindset.
Both libraries come from the same prolific author, Mike Bostock (@mbostock here), who previously worked on visualizations for the New York Times. Mike's new company, Observable (https://observablehq.com), a very thoughtful take on computational notebooks, is also worth a look.
I've been secretly hoping he'll take on Graphviz next.