COMP6700: Lectures Block 5

Alexei B Khorev

April–May 2013

Topics of the Block 5

In this part we shall consider most basic principles of Graphical User Interface programs and Event-Driven programming. Our examples will be (mostly) based on a new set of Java’s API for GUI and Rich Client (also called Rich Internet Applications, RIA) programming — JavaFX.

  1. Graphical User Interface Programming

  2. Asynchronous Control-Flow: Event-Driven Programming

  3. Java Platform for GUI and Rich Client Programming:
    • Abstract Window Toolkit (AWT)
    • Swing (and Java Foundation Classes)
    • JavaFX

  4. Inner classes and their use in GUI

  5. JavaFX and Rich Client Applications

  6. JavaFX features:
    • Layout, Controls, Events and Event Handlers
    • Node Builders: a cute implementation pattern
    • Transition and Animation Effects (easy unlike in Swing)
    • Touch Screen Interface Support: gestures
    • fxml/css: declarative approach to interface programming
  7. Model-View-Controller Architecture and Observer Design Pattern
    • Java’s Observer interface and Observable class

Graphical User Interface

Nowadays, all computer programmes which are intended for interaction with the human user are created with Graphical User Interface (while “In the Beginning was Command Line”, by Neal Stephenson). According to Wiki: “allows for interaction with a computer or other media formats which employs graphical images, widgets, along with text to represent the information and actions available to a user. The actions are usually performed through direct manipulation of the graphical elements.” The history of GUI development so far is very worth reading. Since standard computers have a well defined and constrained (less constrained now, after introduction of iPhone and iPad) interface, the type of GUI which can be deployed on them is often called WIMP (window, icon, menu, pointing device).

One of the very first GUI as it was created by Xerox 6085 (“Daybreak”) workstation in 1985. It inspired the first mass-produced computer with native GUI — Macintosh, by Apple Comp. As Steve Jobs explained regarding this influence by quoting Picasso: “Ordinary people borrow, real artist steals”.

Xerox_6085

Java in the GUI Game

Was there from its birth

Program Architecture

Command-line Application GUI Application
comd_line_app gui_app

Event-Driven Programming

Event-driven applications involve a new type of flow-of-controls — events. Events are asynchronous (happen at unpredictable time), they manifest themselves by altering the state of execution environment. They are detected though this change. Event-driven programs have the following distinctions:

GUI Application events:

Event Handling Models

Polling (old, single-threaded)

Callbacks (modern, multi-threaded)

GUI Concepts

widget

a GUI component like a window, a push button, a label, a scroll-bar, a menu item…

pixel

a single dot on the display screen; displays typically have 800×600, 1024×768, 1280×1024 pixels

depth

the number of bits per pixel

  • depth 1 = black & white
  • depth 8 = 28 = 256 colours
  • depth 24 = 224 = 16,777,216 colours
colour map

translation between pixel values and actual screen colours on a low-depth display

bitmap

an image one pixel deep

pixmap

an image of arbitrary depth (effectively as a 2D array of pixels, although you may not be able to access them like array elements)

font

a mapping from character (or symbol) shapes to bitmaps

event

change in computation environment caused by user or smth else which induces application response. An event has the source (widget which registers it) and the target, an application component which will react to it (change as the result)

event handler

part of application which defines how and what is changed as the result of event

JavaFX: Scene Graph

The JavaFX Scene Graph (modelled on graphical applications and libraries like vector editing tools, 3D libraries, and video games) is the model (see below, in Model-Veiw-Controller architecture) of all graphical objects which exist in an application. It contains information about what objects to display, what areas of the screen need repainting, and how to render it all.

scene_graph-1 scene_graph-2

Individual objects (buttons, shapes, text etc) are represented by leaves which are grouped together in the form of nodes. The scene graph is rooted by a container, usually javafx.scene.Group or javafx.scene.Region which is “embedded” into a scene object (a window). All scene graph is then sent as a stage parameter to the (overriden) javafx.application.Application.start() method as the starting point of execution.

JavaFX: Layout

A set of containers (or panes) for flexible arrangements of widgets within a scene graph:

One container can be nested inside another, and ultimately within a JavaFX Application.

Automating Layout: SceneBuilder

The assembly is automated by using a visual tool like SceneBuilder (a standalone and IDE plugins):

scene-builder

“Drag, drop, align, link” — the layout assisted with the SceneBuilder tool. The created UI is saved in a .fxml-file which can be loaded in the code of a program.

Nodes and Properties

Nodes (containers, layouts) and leaves (control elements, text, indictors like progress bar, and views like scroll views, list views and tree views) — all have properties: shape, size, colour, effect etc. When a node or a leaf element is created, its properties a set either explicitly, or by default; they can be reset as a part of transition/animation effect later during program’s execution.

Text text = new Text("JavaFX technology is kind of cool");
text.setFont(Font.font(“Serif”, FontWeight.BOLD, 30));
text.setFill(Color.GOLDENROD);
DropShadow dropShadow = new DropShadow();
dropShadow.setRaduis(3);
dropShadow.setSpread(0.5);
text.setEffect(dropShadow);
text.setCache(true);
root.getChildren().add(text);

Remember to add a newly created element to its parent according to the scene graph (the last statement above).

Node Builders

A neat JavaFX API trick is builders for creating elements and setting their properties right on the spot. An element (like Circle, FadeTransition or what have you) is created not by a constructor, but by a factory method create() of the corresponding builder class (CircleBuilder etc) which is followed by a chain of property-setting method invocations. Each method returns an object with a modified state. When the method build() is called, the construction is finished and the object is returned to the create-caller:

Rectangle rect = RectangleBuilder.create()
             .width(20)
             .height(200)
             .fill(Color.BLUE)
             .x(400)
             .y(100)
             .build();

RadialGradient grad = RadialGradientBuilder.create()
             .centerX(c1.getCenterX())
             .centerY(c1.getCenterY())
             .radius(50)
             .proportional(false)
             .stops(new Stop(0, Color.RED),
                    new Stop(0.5, Color.BLUE),
                    new Stop(1, Color.GREEN))
             .build();

The builder class definition is defined generically at the level of the class Builder<B> from which every builder class inherits. The property-setting methods return <B> type; the use of reflections is crucial.

Events and Event Properties

When a change occurs (user input etc), the application gets notified by an appropriate event. In JavaFX, event is an instance of javafx.event.Event or its subclass — DragEvent, KeyEvent, MouseEvent, ScrollEvent and others. To define custom event types, extend the Event class.

Event has properties:

Subclasses of Event have additional properties: MouseEvent includes information about which button pushed, how many times, and the mouse position at that moment.

Once an event is detected by some node, it travels through an event dispatch chain, a sequence of nodes connected by their presence on the same scene graph. Standard JavaFX elements have their event dispatch chain already defined.

Event: Propagation, Filtering and Handlers

An event propagates through its event dispatch chain to the target where an programmed action will take place. The event delivery from source to target takes four stages:

  1. Target selection — based on the internal rules, eg for a key event, target is the node which has focus;

  2. Route construction — initially, set by implementation of EventTraget.buildEventDispatchChain() method, but can be modified by event filters along the route process the event (sometimes event can be consumed before reaching its target); in the example SimpleShapesAndTransitions.java, the dispatch chain for the “mouse entered” event is primaryStage --> scene --> root --> circle (c1).

  3. Event capturing — when the event reaches its target (we ignore filters at the intermediary nodes of the chain which may consume the event) and is processed by a dedicated event handler (which executes its code):

    c1.onMouseEnteredProperty().set(new EventHandler<MouseEvent>() {
    @Override
    public void handle(MouseEvent e) {
        ft.stop();
        ft.playFromStart(); // fading the colour of circle
    }
});

  4. Event bubbling — after event is processed by its target, it travels back to the root, and all intermediary nodes, if they have registered event handlers, execute their code too (Ponder: what effects can be realised in such scheme?). To prevent event bubbling from a node up, the method consume() is called.

To learn (important!) details about the event delivery, study the JavaFX Tutorial chapter Events/Event Handlers.

Implementation of Event Handlers: Anonymous Inner Classes

To make an element (node, leaf) to react on event, it should set an appropriate property to be an event handler object which executes the reaction code. This code is usually involves an anonymous inner class — for the moment, a standard Java’s technique to pass around a code to execute. This code (an instance of a state-less class) needs not to be anonymous, or even inner, but the usual call-backs are just that — instances of anonymous inner classes.

final Scene scene = new Scene(root, 600, 450, Color.WHITE);
final Rectangle rect = new Rectangle(...);
// setting rect's properties
// setting rect to rotate once by the angle 45 deg, around the pivot (410,200)
rect.getTransforms().add(new Rotate(45, 410, 200));
root.getChildren().addAll(rect);
/* scene is set to react to pressing "R"-key event; rotation (22.5 deg around 
/  rect's centre) is performed every time the key event is detected */
scene.onKeyPressedProperty().set(new EventHandler<KeyEvent>() {
    @Override
    public void handle(KeyEvent ke) {
        if (ke.getCode() == KeyCode.R)
            rect.getTransforms().add(new Rotate(22.5, 410, 200));
    }
});

An example SimpleShapesAndTransitions.java illustrates event handler use for fading and motion effects triggered by the user inputs.

Transition and Animation Effects

Graphic elements (leaves and nodes) can undergo changes (transitions) including:

The transition and animation effects API are documented in javafx.scene.transform and javafx.animation packages.

The animation effects are generated by synchronising changing scene graph data with their visual representation. The JavaFX rendering engine called Prism generates a special event called pulse (with up to 60 fps frequency) during animation run.

Morphing

An interesting version of animation effects is morphing — making an object to change its shape (with or without moving) and other characteristics (colour, opacity).

Morphing shape (rectangle-to-circle), moving it and changing its colour:

morph-0 morph-1 morph-2

It is not very easy to achieve unless the objects (original “from” and final “to”) are simple enough, like in the example Morphism.java.

One can use Path class and fill it to generate the shapes involved. The morphing would involve calculating the coordinates of path’s points during the transition. In general, this is an interesting project worthy of an assignment, but we’ve already got one, right?

Declarative Programming in JavaFX: UI Layout

The SceneBuilder tool saves a UI as a fxml-file. The fxml format is an XML schema in which XML-tag names match JavaFX class names, such that the content of fxml description file is mapped into the Java code when it’s loaded by the FXMLLoader class (an optional part of any JavaFX application).

<StackPane prefHeight="375" prefWidth="500"
   xmlns:fx="http://javafx.com/fxml"
   fx:controller="steveonjava.Controller">
   <children>
      <ImageView fx:id="imageView">
         <image>
             <Image fx:id="image" url="http://farm1.static.flickr.com/…"/>
         </image>
      </ImageView>
      <Text fx:id="text" cache="true" text="The year’s at the spring,&#10;…"/>
      <Button fx:id="button" text="Play Again" onAction="#replay"/>
   </children>
</StackPane>

Instead of labouring on the layout using Java’s statements, use SceneBuilder to create it and load the resulting fxml-file:

Parent root = FXMLLoader.load(().getResource("app_layot.fxml"));
Scene scene = new Scene(root);

Declarative Programming in JavaFX: CSS styling

Element styling: The fxml-file is complemented by a css-file that defines the styles for the application elements:

#text {
   -fx-font-family: serif;
   -fx-font-weight: bold;
   -fx-font-size: 30pt;
   -fx-fill: goldenrod;
   -fx-effect: dropshadow(three-pass-box, black, 3, .5, 0, 0);
}

#button {
   -fx-background-color: linear-gradient(darkorange, derive(darkorange, -80%));
   -fx-background-radius: 24;
   -fx-padding: 12;
   -fx-font-size: 16pt;
   -fx-font-weight: bold;
   -fx-text-fill: white;
}

The CSS definitions are loaded as follows (following the instructions to load the UI description from a fxml-file created by SceneBuilder):

scene.getStylesheets().add(().getResource("app_styles.css").toExternalForm());

GUI (Rich Client) Programming Key Steps

  1. Choose your interface (how many windows etc)

  2. Choose layout of every container window, fill them with control and display elements

  3. Define (set element properties, define and create event handler objects) what events will be detected and how application will respond

  4. Decide on transition and animation effects

  5. If UI is complex, use SceneBuilder tool to create it and load it fxml-description into application

  6. Decide if elements need adornment and other “rich style” features, use CSS styling and applied them by using resources from css-file

Study JavaFX

The best way to learn GUI programming with JavaFX is to write a JavaFX GUI program. Despite there are already several books on the subject, one can get by by using only two sources:

Official JavaFX Tutorial

A set of presentations supported by good examples about all major aspects of creating (and deploying) a GUI/RichClient application. It contains far more than we need in our course, so pay particular attention to these sections:

JavaFX API Documentation

The standard and indispensable JavaFX API documentations (as usual, often with coding examples which facilitate understanding). Use it! (you can get a local copy to install on your computer).

The Oracle’s JavaFX web site also contains a few video tutorials which you may find useful.

Model–View–Controller architecture

The Model-View-Controller architecture (MVC) was introduced as a part of the revolutionary Smalltalk-80 platform (much like Java + NetBeans but 15 years earlier).

but 15 years earlier). The architecture is based on the concept of separating an application engine (logic + data) from its user interface, so that different interfaces can be used with the same application, without the application knowing about it. The intention is that any part of the system can be changed without affecting the operation of the other. Eg, the way that the GUI displays information could be changed without modifying the actual application or the textual interface. Indeed, an application needs not know what type of interface is currently connected to it. The MVC components are:

MVC

Observer design pattern

When a GUI app is designed with separation of data/logic part (Model) and representation part (View), it’s important to maintain consistency between the separated classes. But for the sake of reusability, the classes should not be tightly coupled. The OO design has a solution to creating a loosely related components (classes), such that a change in one of them (subject, or observable) changes, the others (observers) are notified, and as the result the observer queries the subject for synchronisation of its own state with the subject state. The number of observers is not limited and can change dynamically. The subject does not depend on observers: When the subject changes its state and notifies the attached observers, it doesn’t need to know how many of them are, and what they are. The strong coupling is avoided because the observer does not have to watch the subject closely, instead it gets notified when a change in the subject state occurs. This solution is known as Observer pattern.

obsvr

Observer is a part of the famous Design Pattern catalog created by the so called Gang-Of-Four (E.Gamma, R. Helm, R. Johnson and J. Vlissides) in 1994.

Observer and Observable

The Java API helps to implement the Observer pattern by providing Observer and Observable types (both are in java.util package).

Observable

this concrete class contains several methods

  • public void addObserver(Observer o) (there is also a method for counting registered observers)
  • public void deleteObserver(Observer o)
  • public void notifyObservers() (+ one overloaded with a single Object parameter)
  • protected void setChanged() (there is also a protected method clearChanged())
  • public boolean hasChanged()

An application subject (model) class can extend the Observable class, override the above methods and add more methods (see the Clocks examples).

Observer

this interface declares a single method

  • void update (Observable o, Object arg)

This method is called whenever the observed object is changed. An application calls an Observable object’s notifyObservers() method to have all the object’s observers notified of the change.

Observer and Observable are not used in the Assignment Two starting code “Dots and Boxes”, although do consider employing them in your code.

Simple MVC example: Counter

A simple application with separated Model and its representation:

The Model part is unchanged for three View clients. The Controller part (simple, almost non-existent) is integrated with each corresponding View.

Exercise One

Introduce Observer design pattern and make all views to register; make the client application choose to add and delete observers at run-time

Exercise Two

Modify CommandLineCounter.java to turn it into event-driven client, like the other two (Hint: study a much more complex example in org.xml.sax API; Define a listener interface to read, increment and reset; define all event types)

Clock Example

The Clock application is implemented with Swing, it consists of the following classes:

  1. Clock.java — main class; creates model and view, and registers the latter as an observer to the model; also creates the controller object
  2. Model.java — the application’s Model, maintains and operates with time data; extends the Observable class; in this version, the observable model has one Observer view, which gets notified every time the model state changes
  3. View.java — responsible for representing the time in a graphical form, the application’s View; implements the Observer interface by making update() to repaint the the display panel; registered with model as observer
  4. Controller.java — links model and view, and determines the occurrence of timer events (both timer and panel generate ActionEvents), and defines the event handler (ActionListener) for responding to events generated by timer objects which generates periodic events to trigger change in model’s state
  5. AnalogClockPanel.java — used by View to create a displayable graphical representation of the analog clock; Java2D is used to perform the actual drawing: the model supplies the data about the position of hour-minute-second hands

The Swing model is less clean, especially in the event handler part, where it relies on raw event handler type system (as the result, there are too many different event handlers with somewhat messy system of associations to various events). Exercise: Re-implement the Clock using JavaFX API. Since JavaFX has a rich collection of shape elements, there will be no need to involve explicit drawing on a graphics content (though JavaFX does have ability to get an excess to graphics content of its components and modify their properties).