Tue 10 Mar 2009 (07:03:01)

“Dream Design App”

Imagine — a graphical environment that brings all the little islands of free code together. Yeah right, and cats can be trained to march in single file :)

My dream design app

(A working draft)

The nutshell

Combine Blender, Inkscape & The Gimp. Mix it up with external piped tools and APIs. Stir in some Python for remote control and season with magic.

The way we do things now

Let’s face it, we design something in one app and then we jump through hoops to get it into another. The import/export thing is tacit admission that “our app ends here and another might begin, but we don’t really care.”

We have standards, let’s use them. I can design pretty much all my web graphics in Inkscape, but let there be a jpg or a transparent gif required — well, I’m in for another layer of translation: I’m off to the command line or to the Gimp. Now I have more than the SVG file; I have a few XCF files, some wayward test jpgs, a directory full of conversions, another full of half-written and badly formed scripts. Yuk.

I’m in Blender and I want to design things in 2D and then pop them out. Why can’t I get Inkscape-like tools to do it? Sure I get some tools, but they don’t zing!

I’m in Gimp and I want to do much the same — I want to use vectors, I want to pop into 3D, I want to slip into perspective and tilt something upright — nope. Certainly not rapidly. Certainly not without much planning and masking and ctrl-Z-ing.

Doing graphics means moving from one form to another and back again. Every time you have to “mind the gap” between this app and the next it blocks the flow. All our great open source design apps are being built on islands. I think there are smart people out there who could do better. I think they could join those islands into something like a graphical operating system — not a suite but an environment.

Some fantasy

I want to start with a blank infinite canvas. I want a few little icons dotted about. If am drawing a curve and it approaches them, they scurry out of the way and then regroup. Now I want to change the angle on the whole thing; I pull the page into a new 3D perspective. I carry on drawing on the plane before me. I move to and fro, darting from one side to another, one perspective to another. I am using curves and groups and clones.

Drop some type down; what’s it look like as a city block? Pull it into Z space. Bevel it somewhat. I need a building-like texture on the sides. I open a drawer and browse for some images. I assign a texture; it’s there. I can drag it around on the faces. I can layer it and fade it. I can rotate and scale it.

No texture? I draw something — still in vectors — and assign it to the faces I want. I can use clones too. Change the colours? No problem, everything is styled. Go find the tags and assign new colours.

Let’s put some graffiti on the side. I make a new layer on the plane of the texture and I call-up some reference art. I use my airbrush vector tool; trace out a design which I can tweak; it appears as if sprayed from a can. I remove the guide reference.

Let’s clone that building and moosh it further backwards; the app is generating mipmaps ¹ as it goes. When I am done, all the right images will be there for pulling-in.

I want to use airbrushes and textures now — a more intense and obviously raster method — I call-up the tools; I get an area of focus, a stable plane, a sub-canvas. Off I go; gimping away. Effects are all live, on-canvas, with small intuitive controls that come and go. All done, it merges into the flow again.

I am saving snapshots as I go. One virtual format, many operations. It’s keeping track of the SVG files. It’s saving all the bitmaps and blend files as I go.

When I use fonts, it’s keeping a master style list of them. I can swap them out and style them en-masse as I need to. When I bring-in another drawing needing new fonts, it uses a font manager to temporarily install them. If they are unavailable, I get to decide. Nope, these fonts are not free. Let’s switch them for Century Schoolbook. Need to fetch them? Apt-getting hold on for a moment. If a font is too complex, I can elect to stand-in a simpler one while the other waits in the wings.

Everything is always editable. I can set and flow my blocks of 3D type. I can change my bitmaps and alter my drop-shadows and curves at any point.

I work on an infinite canvas that can seamlessly handle multiple files at once — giving me the illusion that they are all loaded. If I need a part from one I can get at it, drop it into another. Zoom out to overview it all. Drop in, pan around.

I want to make stock parts for an entire set of graphical charts. I draw them and clone them. The clones appear in a drawer to the side. I can split my canvas, Blender-style, in quads and quickly see multiple views of the whole affair. I can flip-book through pages of work too, compiz cube-style perhaps. I employ the clones, darting between their source and their use.

Finally comes the time I need to prep images for a web site. I need ‘em in various flavours. I drop slices over the areas I need — a plane across the camera, seeing through it to my xyz jumble of other planes and objects: my design. I get quick tools coming from the slices; names, id and other meta fields. I get a scattering of action icons that I drop on them: this one is an alpha png, this one a trans gif. That one a jpg of quality 80. I associate the slice plane (as many as I may need) with an output directory ² and I hit the prep button.

Maybe I want a PDF, another slice plane, another prep. A high-res print version? Slice plane and set the DPI to 300.

Some specific ideas

Sheets/Planes are the basic blocks. They can be assigned as patterns for fills, as textures for 3D areas. They can be cloned too. A plane can also be ‘crumpled’ or folded into 3D — becoming an object.

Planes float about in xyz space with a camera, or many, facing from some point.

Each plane is a canvas or a page, that a mixture of vectors and raster work can fill. 3D objects that are drawn on a plane are tied to it (unless they are detached). I see planes as 3D objects really, and so we could draw upon the faces of one, like a complex folded page, simply by keeping it parallel to the monitor.

Planes/Sheets

Design works best on a flat sheet of paper. Turning that paper into xyz space is the easiest way to get perspective. Having many transparent sheets, each at whatever angle you require is the way to build complex images.

If a plane/sheet is turned all the way around, we would see it from behind. This would show all it’s sub-layers in reverse! naturally we should also be able to flip sheets while keeping the frontal view.

Sheet aligning

Sheets can be aligned in many ways. Square to each other, parallel or as normals and other esoteric methods. “Billboards” could be used — always facing the camera, say to keep a logo frontal to things.

Sheet effects

Why not have a sheet that blurs sheets below it. Others that lighten, darken & use gradients etc. Other special effects like decay, etch, grayscale, pixelate, etc. can be employed. Sheets can have a variable influence on other sheets, say, three sheets down.

Sheets could emit light depending on their contents which would control it’s direction and qualities. This could be sent upwards to back-light other sheets and 3D objects. So you could get vectors with a tinge of light on their edges.

Perhaps a sheet effect to flip whatever is beyond it — like a trick with mirrors.

Sheet conversions

So I have an RGBA scene, but I need an 8 bit version of it. Right, I open an 8bit sheet and drop it before my camera. everything that comes through it is converted. I can tweak the settings as I need. Take a snapshot. Done.

Same goes for other conversions, although complexity of things like CMYK may require better thinking than I can employ.

Sheet animations

Beyond the ideas of animated textures and portals (both mentioned below), why not employ timelined tweening. A flash-like system to move herds of things about. You could change a section of a drawing by sending the sheet to another frame number. Or you could produce fully animated output.

Sheet shapes

There’s no reason to be square or rectangular. A Sheet could be be a vector path. It should be defined by nodes and form a clipping region.

I would use the word “gravity” to describe the attraction/association of stuff to a plane. As you move the plane about (or hide it, or deform it) all the objects follow.

Sheet deformations

Some sheets can be like latices in Blender: They exist purely to scrunch other sheets into forms. You could have spheres, cubes, cones, crumpled-paper, or any custom form.

Sheet containment

Sheets can house other sheets. They are given into the gravity of one and stick around — falling into the z-layering of their parent world.

“Portals”

Planes can be “portals” into other files. Why not display a PDF document or an OOO spreadsheet in a plane? It will mirror that file into your design.

Many programs can be portaled. How about graphs and mind-maps; how about reflecting a web page, or a clipped section of one. Whatever code can produce visible output should be able to pipe into a portal. You could write scripts that employ Python or wget, etc., that fetch data from the web and then produce rich markup — which could be reflected in a portal.

You could portal video too — with means to control it. If my Gnu/Linux system can run OOO, Inkscape, Blender and half a dozen other apps all at the same time without a twitch, then I don’t see why one environment can’t employ them all.

Vectors as rasters

It’s not a new idea, but let’s push it. The airbrush tool can be drawing vector curves, which are displayed as brush strokes ³ of the right size/shape/alpha. Any curve can have meta appearances glued to it, like shadows and glows and bevels. Let’s make this scriptable. Drop an effect icon onto the object and it influences the appearance.

Vectors as splines

Why not drag nodes into z-space. We can have wireframes of them coming off the plane (but still bound to its gravity) that appears as an oil painted streak, all glistening and thick, wriggling in space to dive under your typographic headline.

Clipping

Beyond shaped clipping Sheets/Pages, we could have the normal clip/mask paradigm. This should be more flexible in that one can ‘burrow’ into the clip and work on the elements at any time. Having writ this, I don’t see why sub-sheets can’t simply do this job.

Images as frames

I wrote a “water” plugin for Blender years ago; its basic formula was to generate a bitmap per frame. Each frame slightly different to the last so that they looked animated over time. Each frame is drawn by specific code and controlled via widgets.

So, why not make Sheets that change over time? How about a coffee-stain that appears and then gets messier and then fades. Why not an ever more grungy texture, getting more and more messy over time. How about a gradient that changes tones and angles — then you drop it over another sheet (or layer it over a 3D face) and influence the things behind it.

How about an image drawn from all the latest CC-SA images on a spread of websites?

We could plug Python (or others languages) into an API for that. And/or have fast C code.

Physics real and fake

All 3D objects (sheets and polygons on them etc.) can be made to employ their own gravity and repulsion. Objects can be made to resist intersection and meta-blob into one another.

Add to this particle effects — where the particles can come from clones, or a random sampling of a set of clones — and each clone can be another sheet or a sub-object (even a 3D one) from one.

Typography

Setting

Why not use the power of well known tools like La-TeX and HTML/CSS to set the areas of text inevitably required. (I would favour LyX/La-TeX over HTML.) This could be the portal idea in disguise, but it offers all the tools to set the text right there in the thick of things.

Managing

I mentioned this, but a font manager could be built-in to allow flexible work. Where fonts are not available, or are too complex and slow things down, it could replace them with bitmaps or simply fake text (coxxc xc xxo).

When work is prepped to go away from you (to a printer or another machine) then all the fonts required to open the document would be available in the prep directory structure.

Layers

Layers would work within Sheets. Sheets then lie (at any angle) to other sheets. Depending on their properties, they influence those around them. Within a layer we have predictable z-ordering (top down) 3D objects could be set to ‘break’ through all the layers in a sheet — so that a model of a flower could stand-up through layers of vector grasses and fields of type.

Saving the work

All along I have been picturing a directory structure for the “file” that describes the artwork. It would contain all the SVGs and generated bitmaps (mipmaps etc.). It would contain all the fonts required. It would contain all the styles/css and scripts required too. This could all be controlled from within the app, so that you get what you want. Once that’s done, the whole thing could be tarred-up.

The ZUI

Moving through this world would be a combination of the best of Inkscape and Blender. Already mentioned were the multiple views and split screens and flipbooks. Too, the icons that sense approach and your intent and then act/react. There should be no toolbars of the long-solid-bar variety — rather there should be natural, animated, extensions from one tool to its tree of flavours. Colours should be called-up on a keystroke. Dialogs should all be merged-into the flow, not floating above like bugs splished against your windscreen. Small pixel-sized hot points can expand into 3D controls for anything, given the intention of the approach to them.

Zooming is what a ZUI does. You pan and drop-down and zip-up. This is a 3D space, so you can also orbit and such. Whole areas of the zoomiverse can be devoted to sub-functions. Let’s look at the typography section as an example:

Type Zone

Why have a small dialog with too many things crammed-into it? Have a full screen Type-Zone that you hyperspace⁴ to. It shows the current text (or a sample phrase) in full, in all the fonts available. You pan/zoom around this and then manipulate individual entries with the same discrete tools as ever. Kerning, Size, Style are all quickly there for experiment.

Colour Zone

Colour selection would be the same. An entire field of colours with fields of space for scratch areas where you can drop swatches and smoosh them to others to get colours in between. You select them to control your own palette drawers as you need to.

There would be little icons that spawn code to generate colours based on any algorithms we can conjure. Seen a website with great colours? Portal it in, drop the colour-sucker agent onto it and watch it build swatches. Seen an image you want to use to base a theme on? Do the same. Want to make colours from the sounds in a song? Drop the eareye agent on a portal to an ogg file. Generating harmonies, contrasts, pale grays? All via agents.

All these things happen on the infinite canvas and you can drop neat little beacons around so that you can zoom from one to another. Zap you are there, pick a colour or tweak one, zap! you are back at your drawing, in the same tool you left it from.

Big picture

So, what I see is a common set of tools and a zui. A ZUI with ‘portals’ that allow specific app functionality (always using the same widget sets) to emerge in that place. You could get the full use of Blender (design and rendering) at a discreet point in your design. You could zoom into it so that it’s all you see on your screen. When you have drawn your hyper-real orange with a sparkling drop of water running down the side, you zoom back out and dismiss the active portal — leaving the orange among your other design elements: type, zipping lines, blocks of colour, etc.

More than that; you can now use vector tools to draw onto the orange. You would clone a sphere (for simplicity) as a sheet and size it around the orange. Now you use normal Inkscape tools to draw onto that.

If you need the orange rotated somewhat, you grab it and turn it. You don’t have to switch to Blender, edit things, re-render, re-compress, re-fix edges, fuss, fiddle, curse, export, run scripts, import it into Inkscape, fix the XML because the link went crazy, and finally have to adjust your clipping path.

Concluding bits

What I picture is more than a static graphics app. It’s a multi-purpose multi-tooled environment. You can use it to create a single animated gif throbber or an entire presentation or a single design or a web site. It can produce PDF files (of many pages) and a whole range of bitmap types as output. Perhaps other formats as they are needed.

It all out there — little (and not so little) unix tools all waving their empty pipes in the air begging for a little flow.

Well, I can dream can’t I?