Monday, July 29, 2013

Morphic Characteristics

In the last week, I made some changes to the Morphic stuff. Here is a list of characteristics of my Morphic implementation.

  • Every Morph has an owner and a list of submorphs.

  • Morphs do not have a position/rotation/scaling anymore, but a transformation matrix. This matrix can translate, rotate, scale, ... the morph. It is possible to translate, rotate and translate the morph again.

  • Morph transformation is relative to its owner's transformation (transformation matrix multiplication).

  • It is not possible to get or set the current position of a morph. A morph can be transformed, relative to its current transformation. In order to set the morph position, we can reset the current transformation matrix (to identity) and translate it.

  • Morphs do not have a width or height. In the drawOn: method, we have to specify an "outerShape". The outer shape should contain all drawings. It is used for point-in-morph tests (mouse events) and as the clipping region. For example, a button's outer shape should be its rectangular border.

  • Submorphs are clipped at the outer shape.

Monday, July 15, 2013

Morphic-like Interface: first ideas

As a first interface to build user interfaces with Athens, I chose morph composition. Every morph is an object that contains behavior and has the ability to draw itself on the Canvas. Every morph has an owner and a list of submorphs.



The class diagram gives an overview of the current Morphic classes. "AthensHTMLMorphicSurface" and "AthensHTMLMorphicCanvas" are the Athens-HTML classes with Morphic and event support. I already presented some functionality, e.g. "onMouseDown:", in the previous post ("Mouse Event Handling").

"AthensHTMLMorphicSurface" has a bidirectional reference to a world morph. The world morph is the overall owner of all submorphs, i.e. every morph will eventually have the world morph as a supermorph (except for the world morph itself). "AthensMorph" provides basic morph functionality, e.g. submorphs handling and drawing. The method "drawOn: canvas" contains Athens code that we usually write inside a "drawDuring:" block. "drawAll" draws the receiver and all of its submorphs (and the entire transitive closure).

Every morph has a position (translation), rotation and a scaling vector. These values are encoded in "transformation" (an "AthensAffineTransform" matrix) and separate instance variables (not shown in the diagram). At first, scaling is applied, then rotation, then translation. "globalTransformation" stores the complete transformation matrix for the morph, including all transformations of owner morphs. Consider, for instance, that A is owner of B. If A is rotated by 90 degrees and B is rotated by 180 degrees, we expect B to be drawn with a rotation of 270 degrees (relative to A's owner). "globalTransformation" is used for mouse events (convert absolute coordinates to morph coordinates) and for drawing. "AthenHTMLMorphicCanvas"'s implicit transformation is set to "globalTransformation". Every time we draw something (inside "drawOn:"), the implicit transformation is applied to the path transformation. I will probably implement this differently, later: it is faster to provide a modified "AthensAffineTransform" as path transform that loads "globalTransformation" with "loadIdentity" and applies "globalTransformation" automatically when setting the matrix values.

"AthensDummyWorldMorph" is just an implementation detail: Morphs might not have an owner directly after creating them while it is still allowed to set transformations at this time. This triggers a redraw automatically and also requires the owner to be redrawn (there might be something behind the morph that now becomes visible). Of course, this does not work if the morph does not yet have an owner and was never drawn yet. "AthensDummyWorldMorph" is a null object.

There are some issues that I could not solve yet: for example, when transforming (e.g. rotating) a morph, I have to redraw the whole world because something behind the morph might become visible. This could involve a lot of rendering, but I cannot think of an easy alternative.

Step35 and step36 of the tutorial contain examples for composing and transforming morphs. You have to run step33 before, because it creates the morphic surface.

Mouse Event Handling: first ideas

For building GUI applications with Amber-Athens, mouse events are very important. HTML Canvas itself does not provide any mouse event handling, but we can use JavaScript elements (as we can on any DOM element). We can define a callback that gives us the pixel position as a parameter for mouse movement, mouse down and mouse up event.

I implemented the even-odd rule to determine if the user clicked on a shape. This only works with polygons, therefore we need to convert curves first (not implemented yet). A path might consist of several polygons (due to "moveTo:"). In that case, we run the algorithm for every polygon ("contour").

This does, however, not work as expected if two shapes intersect each other. In this case, we might want to check if we clicked the upper or the lower shape. The solution is to do the point-in-polygon tests in the inverse rendering order.

Amber-HTML-Morphic contains subclasses of AthensHTMLSurface and AthensHTMLCanvas that provide event handling functionality and some first Morphic ideas (see next post). The idea is to separate the Athens code from event handling and Morphic stuff, such that AthensHTMLSurface and AthensHTMLCanvas only contain code for rendering graphics. Since event handling is platform-specific, we need separate HTML and Cairo implementations.

We can bind events with "onMouseMove: aBlock", ..., just like DOM events in Amber. Step34 in the tutorial shows an example.
	|path poly|
"Step 34: Event handling (just a piece of the code)"

surface drawDuring: [:canvas |
path := canvas createPath: [:builder | "Create path"].
poly := path asPolygon].

surface onMouseMove: [:e | surface drawDuring: [:canvas |
surface clear: Color gray.

canvas setShape: path.
(poly includesPoint: e offsetX @ e offsetY)
ifTrue: [canvas setPaint: Color blue]
ifFalse: [canvas setPaint: Color green].

canvas draw]].

The code above shows a part of the code in step34. "path asPolygon" converts a path to a Polygon. At the moment, this works only for simple paths without curves. A path is a list of operations (e.g. "lineTo:", "moveTo:", "curveVia:", "close", ...) that can be passed to an interpreter.

We have three types of path-related classes in Athens-HTML: "AthensSimplePathBuilder" (part of Athens-Core) generates a path object (that is platform-specific, e.g. "AthensHTMLPath"); "AthensHTMLPath" is the Amber-specific path object that can draw the path by replaying/sending the path commands to itself; "AthensPolygon" is a polygon that can be built by replaying/sending the path commands to an empty instance of it (from "AthensHTMLPath").

In our case, the path is passed to a new "AthensPolygon". This class will also be responsible for flattening curves to polygons. It also provides a method for the point-in-polygon test. If we receive a lot of mouse events, we should cache the polygon when drawing the path (as we did in the example).

Friday, July 12, 2013

Status Update: Week 4

Athens is almost completely implemented in Amber. There are still some minor issues.

  • Browser compatibility: e.g. Firefox does not support line dashing, "miterLimit" is not set correctly in Chrome.

  • Drawing text: "getPreciseAscent" is not supported by Canvas. There is no nice way to get the height of a text (only dirty hacks involving reading pixels or creating DOM elements).

  • Canvas limitations: some paint modes not supported by Canvas (e.g. "colorBurn").


For now, I will not work on these issues.

The next steps will be implementing mouse events and tests. For mouse events, I have to implement an algorithm to transform paths to polygons and test if a given point is inside the polygon (probably De-Casteljau and Even-odd rule; a part of this is already implemented in Athens).

It is difficult to write automatic tests for Athens. My first idea was to render images and calculate the hash of the pixels. However, this value differs among browsers. I will probably start implementing manual tests: the tester sees a reference image and has to compare it to the actual Canvas rendering.

Athens Performance on Amber

It is no surprise that the Athens implementation in Amber is much slower than the implementation in Pharo. Athens comes with a demo application that repeatedly draws a complex vector image (consisting of many paths) and shows the frames per second (fps).



In the tutorial in Amber, you can start this benchmark by executing step32 (remember to execute step2 before!). When you click "Do it" a second time, the execution stops and print the fps to the Transcript (printing to the Transcript does currently not work correctly in my Amber version).

If you want to run the demo in Pharo, you have to execute "VGTigerDemo runDemo".



On my computer, I have 30-33 fps in the browser (Chromium 24) and 70-80 fps in Pharo. The demo is slower in Firefox (20-25 fps). I ran the benchmarks on Ubuntu 12.04.2 on a computer with an Intel i5 dual core CPU.

Athens-HTML Architecture

In this post, I will describe some important classes of Athens that are used in the Pharo implementation (Athens-Cario) and the Amber implementation (Athens-HTML).

In the diagram below, you can see that there are three kinds of packages (the diagram contains only some packages, classes and methods). Athens-Core-* packages contain classes that are shared among all Athens implementations. For example, AthensCanvas contains functionality for setting the paint and the shape that should be drawn. It also contains a "draw" methods that delegates the drawing process to the paint/shape using double dispatch. The actual drawing code is located in the Paint classes. There are Paint classes for Athens-Cairo and Athens-HTML. The Paint classes in Amber-Core are abstract, i.e. they only define the interface and all methods are subclass responsibility.



At the moment, it is not possible to share Athens-Core classes between Pharo and Amber Smalltalk. Amber has some limitations, e.g. selectors may not contain underscores and class variables are not supported (only class instance variables). However, I could copy most of the code of the core classes.

Project Description

This summer, I will port Athens, a vector graphics library for Pharo, to Amber Smalltalk, as part of my Google Summer of Code (GSoC) project. The project mentors are Nicolas Petton and Igor Stasenko.
What is Athens and Amber Smalltalk?

Athens is a vector graphics library for Pharo that uses libcairo. Athens supports drawing various shapes such as rectangles and paths. Paths can consist of lines and curves (e.g. Bezier curves). Shapes can be drawn with a stroke color and a fill color (paints). The fill color can be a solid color, a color gradient (radial and linear), a bitmap, or another surface (i.e. another drawing). The shapes and paints that I mentioned before, are only the predefined ones that are supported out of the box. You can add your own shapes and paints that use already defined ones. If you are interested in Athens for Pharo, you can find it on Smalltalk Hub.

Amber is an implementation of Smalltalk that runs entirely in the web browser. It compiles Smalltalk code to JavaScript. Amber supports writing JavaScript code, as well. This can boost the performance (e.g. Amber has to wrap all message sends in Smalltalk to support things like "doesNotUnderstand". When writing JavaScript code directly, this is not the case). At some points, I made use of JavaScript inlining.

I will port Athens to Amber Smalltalk using the HTML5 Canvas. The Canvas provides methods for defining shapes (e.g. rectangles and paths) and filling them. Some browsers implement the Canvas with libcario. Most concepts from Athens can be mapped directly to the Canvas. There are, however, some limitations in the Canvas.

  1. The Canvas specification does not support all features that are supported by libcairo and Athens. For example, Canvas supports only 12 "globalCompositeOperations", whereas Athens supports more. There is no easy way to implement the missing ones, so we will skip these for now and implement them, once there is browser support.

  2. Some browsers do not implement the entire Canvas specification or implement it wrong. For example, some browsers support all 12 "globalCompositeOperations" in theory, but they do not draw them correctly. Another example is line dashing in paths: this feature is currently not supported by Firefox. Therefore, it will not work in Firefox (my current implementation simply draws a non-dashed line in Firefox).


My vision is to provide a common vector graphics library for Pharo and Amber Smalltalk. Afterwards, we can use Athens to draw user interfaces in Pharo and Amber Smalltalk with the same Smalltalk code. This allows us to write Smalltalk applications that run both in Pharo and in the web browser.

I prepared a schedule in my project proposal. You can take a look at it in order to get an overview of the next implementation tasks. The timeline is, however, outdated. You can track the project progress in this blog and on the demo website. Athens comes with a tutorial that explains the API. The demo website contains the whole tutorial. It is important, that you always execute step 2 before executing any other step because this step creates the Canvas surface.

Please feel free to write me an email or post in this blog if you want to share your ideas (concepts, implementation, API, or whatever), report a bug, or just want to leave a comment.
Important Links

Source Code on Github
Demo Website
Project Proposal