Ollin

Input

Pointer and keyboard input are plain properties and overridable methods on the sketch.

Contents

mouseX / mouseY

mouseX: Double
mouseY: Double

The cursor position in sketch coordinates (points, top-left origin, y-down). When the window opens, both values start from the cursor's actual position. So a mouse-driven sketch responds on the first frame instead of reading (0, 0) until the first move. After that they follow the pointer as it moves over the canvas.

override func draw() {
    background(.white)
    fill(.black)
    drawCircle(mouseX, mouseY, 24)   // a dot follows the cursor
}

Two calls remap the mouse into their own coordinates. One is withViewBox, which does it for the length of its block. The other is viewControl, which does it for the rest of the frame. The remap keeps a sketch written for the whole canvas working when the view is not the whole canvas. In both cases, mouseX and mouseY give the position in the content under the pointer, not the position on the screen. The pointer values are restored before the next frame.

mouse / previousMouse

mouse: Vector2
previousMouse: Vector2

mouse is the same cursor position as one point. The geometry calls take a point directly, so you can pass it without unpacking it. previousMouse is where the cursor was when the previous frame drew.

drawLine(previousMouse, mouse)          // ink follows the pointer
let speed = (mouse - previousMouse).length
if circle.contains(mouse) { … }

mouse - previousMouse is this frame's drag, so you do not need to track the motion yourself. On the first frame previousMouse equals mouse, so the first delta is zero rather than a jump from the corner. previousMouse holds the canvas-space position the window reported. The coordinate remaps above apply to mouse for their frame, so they never change previousMouse.

mouseIsPressed

mouseIsPressed: Bool

Whether a mouse button is currently held over the canvas. It is the polling counterpart to the one-shot hooks below, and it works the same way as keyIsPressed. Poll it in draw() for anything that should continue while the button is down, such as dragging, painting, or steering. mouseX/mouseY keep updating through the drag.

override func draw() {
    if mouseIsPressed {
        drawCircle(mouseX, mouseY, 12)   // paint while the button is down
    }
}

mousePressed / mouseReleased

mousePressed()
mouseReleased()

Override these to respond to a click. Each fires once per press or release, and mouseX/mouseY hold the event's location. Use mousePressed() to regenerate an otherwise static sketch on demand. Use mouseReleased() to act on a finished drag or stroke. The DigitReader example classifies its drawing in mouseReleased().

override func mousePressed() {
    randomSeed(frameCount)   // re-roll the randomness on each click
}

pressure / pressureIsAvailable

pressure: Double            // 0...1, 0 when nothing is held
pressureIsAvailable: Bool   // whether this device can measure it at all

How hard the pointer is pressed. On a pressure-sensing device (a Force Touch trackpad, a pen tablet) the value varies continuously through a press. On a device that cannot measure pressure the value is 1 while a button is down. A pressure-driven sketch still works there, but only at that single level.

pressureIsAvailable tells you which kind of device you have. The answer comes from the event, not from the machine, so it is false until the first press. Read it in mousePressed() rather than in setup():

override func mousePressed() {
    brush = pressureIsAvailable ? .pressure(light: 0.1) : .speed(fast: 0.15)
}

Ollin asks the trackpad for the drawing gesture, which is a single stage over the full range. A press therefore reads as a smooth amount, and no force-click fires look-up in the middle of a stroke. See Marks for the brush that reads pressure.

stylus

stylus.tilt: Vector2            // how far it leans, -1...1 each axis, .zero upright
stylus.twist: Double            // how far the barrel is turned, radians, 0..<2π
stylus.isEraser: Bool           // the end on the tablet is the eraser
stylus.isNearby: Bool           // a stylus is over the tablet, hovering or drawing
stylus.tiltIsAvailable: Bool    // whether a lean has ever been reported

What a tablet says about the stylus beyond where it is and how hard it is pressed. How hard is pressure rather than a member here, since a trackpad measures that too and no trackpad has a lean.

tilt is the pen leaning away from upright: x positive when the top of the pen leans right, y positive when it leans down the canvas, and the vector's length is how far from upright it is, near 1 when the pen is nearly flat. It is what a broad-edged nib is made of, since a chisel laid across the lean draws thick one way and thin the other:

let angle = stylus.tiltIsAvailable ? atan2(stylus.tilt.y, stylus.tilt.x) + .pi / 2 : .pi / 4

twist moves only on an art pen. isEraser is the end being used, which a stylus reports by arriving as a different pointing device when it is turned over. isNearby covers hovering as well as drawing, so a cursor can be shown under a pen that has not touched down yet.

tiltIsAvailable is the same kind of answer pressureIsAvailable is: it comes from the events rather than from the machine, so it is false until a pen has been used, and it latches on once one has, since a stylus lifted out of range would otherwise take the pen path away in the middle of a stroke. Everything else is zero and false under a mouse, so a sketch written for a stylus still runs there.

A stylus's values are recorded into a take and replayed with the rest of the input. See Marks for the brush that reads a press, and Examples/Input/Pen for a nib that reads the lean.

scrollDeltaY / mouseScrolled

scrollDeltaY: Double   // how far the wheel moved this frame, 0 when it did not
mouseScrolled()        // fires once per wheel event

How far the scroll wheel, or two fingers scrolling on a trackpad, moved this frame. Every movement since the last frame is added in, so none is lost between frames. Positive is a scroll up. Inside mouseScrolled() the value is that one event's movement, and by the next draw() it is the whole frame's.

var zoom = 1.0

override func draw() {
    zoom *= pow(1.02, scrollDeltaY)
}

viewControl and cameraControl zoom with the wheel already.

pinchScale / turnAngle / fingersPinched / fingersTurned

pinchScale: Double   // how much two fingers changed the size this frame, 1 when still
turnAngle: Double    // how far two fingers turned this frame, radians, 0 when still
fingersPinched()     // fires once per pinch event
fingersTurned()      // fires once per turn event

Two fingers on the trackpad. pinchScale is a factor: 1.1 when the fingers spread a tenth farther apart this frame, 0.9 as they close, 1 while nothing pinches. Every pinch since the last frame is multiplied in, so multiplying a size by it follows the fingers exactly. turnAngle is the turn in radians, positive clockwise on the screen, which is the way rotate turns a positive angle. So adding it to an angle and rotating by that keeps a shape under the fingers:

var size = 300.0
var spin = 0.0

override func draw() {
    size *= pinchScale
    spin += turnAngle
    translate(width / 2, height / 2)
    rotate(spin)
    drawRect(Rectangle(center: .zero, width: size, height: size))
}

Both are this frame's change, the way mouse - previousMouse is, so the sketch keeps the totals itself. The hooks fire once per event, and inside them the value is that one event's change. Use them for a response that comes in steps, such as a dial that clicks every fifteen degrees:

var notch = 0.0
var petals = 7

override func fingersTurned() {
    notch += turnAngle
    if notch >= .pi / 12 { notch -= .pi / 12; petals += 1 }
}

viewControl zooms with a pinch about the pointer, and cameraControl dollies with one: spreading the fingers to twice the distance brings the camera to half its distance. When the phone shows the sketch (PhoneDevice.show), two fingers on its screen pinch and turn the sketch the same way. A mouse has no pinch, so pinchScale stays 1 and turnAngle stays 0 under one.

A pinch and a turn are recorded into a take and replayed with the rest of the input. A performance that zoomed and turned then renders offline the way it was played. Examples/Input/PinchAndTurn sizes and spins a rose and clicks its petal count with the turn.

key / keyCode / keyIsPressed

key: Character?
keyCode: KeyCode?
keyIsPressed: Bool

The most recent key event, split into two properties. A key that produces a character arrives as key, so a letter, a digit, punctuation, or the space bar reads as "a", "5", or " ". A key with no useful character arrives as keyCode instead, so you match a named value rather than a magic number. That means an arrow or a function key reads as .leftArrow, .return, .escape, or .function(1). Exactly one of the two is set for a given event, and the other is nil. Both are set on press and release, so the hooks below can read which key fired.

keyIsPressed is true whenever any key is held.

override func draw() {
    background(.white)
    if key == "r" { background(.red) }        // a character key
    if keyCode == .leftArrow { /* … */ }      // a named key
}

The named keys are upArrow / downArrow / leftArrow / rightArrow, return, enter, tab, escape, delete, forwardDelete, home / end / pageUp / pageDown, and function(_:). return is the main Return key, and enter is the keypad one. delete is Backspace, and F1 is .function(1).

keyPressed / keyReleased

keyPressed()
keyReleased()

Override these to respond to keys. Each fires once per physical press or release, and auto-repeat does not fire keyPressed() again. key/keyCode hold the key that fired. The hooks are for discrete actions, so for a continuous response while a key is held (steering, movement), poll isKeyDown(_:) in draw() instead.

override func keyPressed() {
    if key == " " { paused.toggle() }       // space toggles
    if keyCode == .escape { reset() }
}

isKeyDown

isKeyDown(_ character: Character) -> Bool
isKeyDown(_ code: KeyCode) -> Bool

Whether a specific key is currently held. It is the polling counterpart to the one-shot keyPressed() hook, for movement that should continue while a key is down. Call it from draw().

override func draw() {
    if isKeyDown(.leftArrow)  || isKeyDown("a") { x -= speed }
    if isKeyDown(.rightArrow) || isKeyDown("d") { x += speed }
}

The character form is case-sensitive, so isKeyDown("w") and isKeyDown("W") differ by whether Shift was down at the press.

moveAxis

moveAxis: Vector2

The held movement keys (WASD and the arrows) as one direction. Each axis is in -1...1, in canvas orientation, so up is (0, -1) and right is (1, 0). Opposite keys held together cancel to zero.

position += moveAxis * speed * deltaTime

A 3D sketch reads the same value and maps y onto its own forward direction. The usual choice is for canvas up to mean "ahead", so -moveAxis.y is the throttle.

typedText / textTyped / composingText

typedText() -> String
textTyped()
composingText: String
caretPosition: Vector2?

The keyboard read as text, the way a text field receives it. key reports the keys themselves, so Option-E then E is two presses of e. typedText() reports what those presses typed once the system's input methods composed them: one "é". A character picked in the emoji viewer (Control-Command-Space) arrives as itself. Chinese or Japanese typed through an input method arrives once a candidate is chosen. Holding a key repeats it, as it does in a text field, even though keyPressed() fires once per press.

typedText() returns the text typed since the last call, oldest first, and reading it empties it. textTyped() is called each time text arrives, so a sketch can take it there or read it in draw(), whichever it prefers. Return arrives as "\n" and Tab as "\t". Delete arrives as a backspace character, "\u{8}", which append(typing:) reads as taking the last character away:

var caption = ""

override func draw() {
    background(.white)
    caption.append(typing: typedText())
    fill(.black)
    textSize(64)
    drawText(caption + composingText, 60, 200)
}

While an input method is still composing, composingText holds the reading it shows: the underlined hiragana before a kanji is picked, or the accent a dead key holds until the next key. It is empty the rest of the time. Draw it after the typed text, in a color or with an underline of its own, the way a text field does. Once the input method confirms, the text arrives through typedText() and composingText empties. Set caretPosition to the bottom of the line being typed, in canvas coordinates. The input method then opens its list of candidates just below it. Left nil, the list opens below the pointer.

Text reaches the sketch while its canvas holds the keyboard (see Keyboard focus in the hosts). Losing the keyboard drops a composition half written, as a text field does. A take records typed text and compositions, and a replay repeats them. A headless export types nothing. Examples/Input/TypedText puts what you type on the canvas and shows the last key beside the last text it typed.

droppedFiles / filesDropped

droppedFiles() -> [String]
filesDropped()

Files dropped on the window from the Finder. droppedFiles() returns the paths dropped since the last call, oldest first, and reading them empties the list. filesDropped() is called once at each drop, after the paths have arrived and with mouseX/mouseY at the drop point, so a sketch can respond there or poll in draw(), whichever it prefers. Anything the Finder can hand over arrives: a picture for loadImage, a clip, a font, a table. A drop is live input outside a take, neither recorded nor replayed.

override func filesDropped() {
    for path in droppedFiles() {
        if let picture = try? loadImage(path) { pictures.append((picture, mouse)) }
    }
}

Every host takes the drop wherever the sketch is running: the live window, the gallery, and the performance stage with its code hidden. With the code showing, the editor over the stage takes the drop as text instead.

chooseFiles / chosenFiles / filesChosen

chooseFiles(withExtensions extensions: [String] = [], allowsMultiple: Bool = false)
chosenFiles() -> [String]
filesChosen()

Files the person picks through the system's open panel, for a sketch that asks for a picture or a table instead of waiting for a drop. chooseFiles shows the panel over the sketch's window as a sheet and returns at once, so the frames keep coming while it is open. When the panel closes on a pick, the paths join chosenFiles() as String paths, oldest first, and filesChosen() fires once. Reading chosenFiles() empties the list, as droppedFiles() does. A panel closed without a pick adds nothing and fires nothing.

extensions limits what the panel offers, written without the dot ("png", "csv") and in any case; empty, the default, offers every file. An extension the system has no type for still matches files by name. allowsMultiple lets the person pick several at once. While a panel is open, a second call does nothing.

var photo: Image?

override func keyPressed() {
    if key == "o" { chooseFiles(withExtensions: ["png", "jpg", "heic"]) }
}

override func filesChosen() {
    for path in chosenFiles() { photo = (try? loadImage(path)) ?? photo }
    redraw()   // a still sketch shows the new picture
}

A pick is live input outside a take, like a drop: neither recorded nor replayed. The panel is a Mac window; on a phone or a tablet, and in an export, the call does nothing.

pointerShape / hidePointer / showPointer

pointerShape(_ shape: PointerShape)
hidePointer()
showPointer()

The pointer over the canvas. pointerShape gives it one of the system's shapes. hidePointer() takes it away, for a sketch that draws its own (a brush outline, a reticle) at mouse. Both hold until changed, and both apply only over the canvas. The rest of the window, and a host's sidebar beside the canvas, keep the ordinary arrow.

The shape and the hiding are kept apart. A shape set while the pointer is hidden is the one showPointer() brings back. So a sketch can pick its next shape without showing the pointer early. A call that changes nothing costs one comparison, so a sketch can set the shape in draw() every frame from what is under the pointer:

override func draw() {
    let overDisc = (mouse - center).length < 80
    pointerShape(overDisc ? .openHand : .crosshair)
    drawCircle(center: center, radius: 80)
}

PointerShape names the system's own cursors: .arrow (the default), .crosshair, .pointingHand, .openHand, .closedHand, .iBeam, .notAllowed, .zoomIn, .zoomOut, .columnResize, .rowResize, and .dragCopy.

An installation that hides the pointer (Installation.hidesPointer) hides it over the whole screen whatever the sketch asks, and the sketch's own hiding never undoes it. On a phone or a tablet there is no pointer, and in an export there is no window; both calls do nothing there.

Keyboard focus in the hosts

A window that is the sketch gives the sketch the keyboard as soon as it opens. That covers a standalone swift run and the live host, so everything above works with no click first. The examples gallery is different. Its example list keeps the keyboard, so the arrow keys navigate the examples. In the gallery, a sketch that reads keys shows a small "Click the sketch to use the keyboard" prompt over the canvas. Clicking the sketch gives it the keys, and clicking back in the example list returns them.

When you embed SketchView in your own SwiftUI app, the keyboardFocus: parameter sets the same policy. .automatic is the default, and it claims the keys when the view appears. .onClick leaves the keys to the rest of the window until the canvas is clicked. showsKeyboardHint: shows that same prompt for sketches that want keys:

SketchView(sketch, keyboardFocus: .onClick, showsKeyboardHint: true)

Typed text follows the keys: whoever holds the keyboard gets the text. On the performance stage the editor holds the keys, so what you type goes into the code, and a click on the canvas hands both to the sketch.

↑↓ move ↩ open